Knowledge is extremely vital for survival. The second most essential element you need for a survival scenario is a good level of fitness.
If you're just hiking and camping then you just need to have the conditioning necessary to walk for a mile or even a few miles. Obviously, if you intend to be the truest of survivalists then you'll probably need condition yourself to be able to walk 10 to 20 miles a day. I believe a very good amount of strength should be added to that "true survivalist" equation as well. The survivalists should learn to love Strength and Conditioning.
I can't tell you what your goals should be but I'll tell you not to delay on achieving them. There's no time like the present. Enjoy your life.