Welcome to the FZ forum Dustin.
I think I'm the right guy to answer some of your questions. I own both an FZ6 and DRZ400, and a good friend owned an XT225 and I got enough time on it to know what I think about it.
The XT is a great beginner bike, but when compared to a true off-road bike it's not a very capable bike. I'd call it a "play bike". I think for gravel roads and fire trails it's a fine machine, but it quickly falls on its face on any type single track or challenging hills.
The DRZ comes in a few different flavors. There are the E & S models that are dual-sport bikes (some E's are off-road only...depending on the year) and the SM that is strictly a road bike. The early E's make the most power (more agressive cams, pumper carb, higher compression ratio) where as the S and SM are choked by EPA regulations. As an off-road bike, the -400S is OK. It's heavy and geared as a dual-sport. I think it does better in desert environments than single track. It's perfectly fine for fire roads and gravel. I use to race off-road (hare scrambles and some MX) and a DRZ wouldn't be my 1st choice, or 10th choice, for an off-road machine (I don't like dual-sport bikes...they suck at everything). But if you need to be able to ride from trail to trail on public roads, the DRZ is a good choice. They have been in production for over 20 years and there is a wealth of knowledge on ThumperTalk (there is a dedicated DRZ forum). My DRZ is an E model that has been converted to street use only. I have significant modifications to the suspension and brakes. It is the most fun motorcycle I've ever ridden on the road - period. It's not happy on the highway, and it sucks for two-up riding - but get it on a twisty backroad and you'll never get the smile off your face. That's it's element. My summary; DRZ is a great dedicated backroad bike, an OK dual-sport (fire trails/gravel), and not such a good bike for technical off-road riding. There are some known issues with all year/model DRZ's that are well documented on ThumperTalk including some lock-tite fixes (internal to the engine), carb jetting (it is an analog bike) and stator/charging issues. But like any motorcycle, a well maintained machine will be reliable.
The FZ6 (2004-2009) is a great sport-standard motorcycle. It's comfortable, somewhat light-weight (450 lbs), has good power (around 100hp) and has real good EFI. It's a nice bridge between analog motorcycles and the new generation digital bikes. There are no frame cracking issues that I'm aware of on the FZ6. In fact there aren't any real issues for a well maintained machine - and it's damn-near maintenance free. A common issue that pops-up on machines that aren't ridden often is the in-tank fuel pump. The suspension is the weak point of the FZ6 (with almost all Yamaha FZ series motorcycles) but there are several things that can be done to improve the OEM suspension...and some owners have swapped in suspension components from other motorcycles. But out-of-the-box, the FZ6 is a great first street bike for someone that already knows how to ride. It can be modified as your experience grows. I've been riding motorcycles since I was 10 years old and have owned just about every type/style bike made - and I really like the FZ6 for it's balance of performance, comfort and reliability. As you mentioned, good (and real good) examples can be found in the $3,000 range.
A $3,000 FZ6 is going to be a nice bike. A $3,000 DRZ is gonna be a rat and probably in need of some investment. That should be a consideration as well.