Most states offer up datasets of their unpaved roads.
After some googling around I did find a dataset covering gravel roads in Pennsylvania:
They do not offer their data in GPX format but they do support the ESRI Shape File format which is included in the zip download.
ESRI Shape files can be converted to GPX using GPSBabel.
It's pretty large at 63MB so too big for the prototype mapping tools I have here on the site. But it should load (albeit slowly) once I get the new editor fielded Any Day Now(tm), which I've been saying for something like I year.