Question about retainers and situation resulting from not wearing them?

Hi. I just went to the orthodontist today, pretty sure that my teeth wouldn't be that bad after not wearing the retainer continually every night, but when I got there, after an examination, the doctor told me that the teeth have shifted a lot and basically nullified the effect of braces 2 years prior, and that the only way to fix the damage was through retreatment. Now, I realize that I made a huge mistake in not wearing them nightly, and I acknowledge that it's basically my fault for what has happened to my teeth. However, do I really need retreatment? I can't bear to go through braces again, and the price of Invisalign really doesn't justify it. If I continue to wear the retainer, however tight and perhaps useless it may be, will my teeth return to the previous state as the retainer is pulling on them? Or will I really need retreatment to fix the damage. Thanks for your input.