Nicking assets from Tachyon: The Fringe, Part 2: The PFF Archive

The first file we’ll look at on our foray into pixel piracy is the PFF Archive. This is an archive file containing all of the game assets for Tachyon. There’s just the one, Tachyon.pff, and it’s nearly 400MB in size (no compression, but when extracted the contents usually take ~2GB on disk, due to a lot of small files). (This was back in the day when 2GB was a lot). This file format has been used by several NovaLogic games, most prominently the Delta Force series. Later versions of the format seem to support compression, but the format we’re working with is basically straight storage.

Technically the PFF format isn’t vital for our model extraction project- extractors have been around for the past 20 years, and we could just as easily work off an already extracted archive. However, that makes moving between computers a bit annoying, as you then need to find the extractor, extract the directory, point the app at the directory. Too much work, easier to point the app at a single file (especially as we aren’t looking to modification of underlying files). This also serves as a starting point for the blog series, to play with formatting and such before the content is important.

Thankfully at least one of the PFF extractors (from Devil’s Claw) also came with source code; so no reverse engineering required, just a minor port. Thank you Devil’s Claw.

On with the show.

 

 

General format of the file is a Header, a set of File Entries, a Footer, and then a massive blob of data.

 

Header

Size	Type	Name	Value	Comments
4	int	Header Size	20
4	char[4]	Version String	PFF3	PFF3, PFF4, etc
4	int	File Count	8134	Number of File Entries
4	int	File Segment Size	36	Size of Individual record, 32 to 40 bytes
4	int	File List Offset	0x16C6A7F1	Start of File Entry Array, near end of file in this case

One interesting bit about the header is that the version string is lies (according to the old code). File Segment Size varies by PFF version, from 32 bytes for “V2”, 36 for V3, 40 for V4. But, “V2” and V3 have the same version string: PFF3.

File Segment Entry

Size	Type	Name	Comments
4	int	Deleted	0 for “not deleted”
4	int	File Position	Index into PFF where file content starts
4	int	File Length
4	int	Creation Date	Guessing unix like timestamp; all around 2000 if true
16	char*	File Name	Null terminated string
4	int	Modified Date	Optional based on Segment Size, present in our data set, and has some weird values for timestamps (2020s+)
4	int	Compression Level	Optional based on Segment Size

Footer

Size	Type	Name	Comments
4	int	System IP	Who packed the file, a 192.168.*.* address.
4	int	Reserved/Unknown
4	char[]	KING tag. Value=KING… Magic value, at a guess?

Appears after all of the File Entries. At a guess the KING tag is intended as a magic value to check that the file is otherwise valid.

Thoughts

It’s a straightforward archive format, with some support for extensions built in. It’s not bad, and kind of elegant for the goal of ‘single file archive’. It’s a bit wasteful though- what I usually see, when size matters, is the entries will have a bit field, with some fields defined, and others reserved for future use. IE deleted and compression level don’t need 8 bytes all to themselves. Though I’ll grant, for this use you’re talking about 8 bytes across 8000 entries- 64KB of semi-wasted space in a 400MB file.

Source code for the C# reader can be found here, and the Source for the Devils Claw extractor is here.

Other Notes: I wrote this a long time ago and never got around to publishing it because I wanted to get formatting for the tables right. Giving up on that for now, and just publishing to get the process started again. This’ll be a bigger problem in some of the next file formats, unfortunately, but we’ll see what happens. I also may use this as an opportunity to learn Rust, as Visual Studio is getting on my nerves.

Nicking assets from Tachyon: The Fringe, Part 1: Why and What

Intro

Many of the projects I’d like to do in my non-existent off time involve game development. I want to create an RTS, or a Fighter sim, or similar, but I inevitably run into a problem that’s always been a stumbling block: All these projects at some point call for shiny 3D assets. And I’m a programmer, not an artist. How can I get 3D assets to start development with?

I could look for free assets online, but I always find that kind of hit and miss. You have to spend hours upon hours to find a few good models, all in disparate formats that then need markup and corrections. It’s a mess that zaps my desire to live and my desire to do development.

I could pay someone money to generate models and textures for these games that likely won’t get past the weekend project phase. This still has the problem that I’m not an artist, so the requests would come down to “make me a cool looking spaceship”. That’s destined for disappointment.

OR. I could nick assets I already like from a game.  And not just any game, but the best game ever: Tachyon: The Fringe.

Disclaimer: In case you can’t tell, these are soley for personal non-commercial projects. It’s me trying to ease the path to getting something interesting on screen so that I can keep momentum up.

Starting Info

Tachyon, the best game ever, is a Space Combat Simulator released in early 2000. It’s actually still available today via Steam, though I’m sure no one at Nova Logic has touched the source code in the past 12 years. There’s been one notable project to try and modernize it, the fan made FringeSpace total conversion for Free Space 2. As of this writing that’s still ongoing (and I think I read they’re in their 6th or 7th year). One of their first projects was extraction of model data, but for whatever reason they had a lot of issues with that, and instead went the route of re-creating the assets by hand.

The original attempt at reverse engineering the model formats is semi-documented here (see Development Log, mid way through, apparently a copy of a LiveJournal of a fellow named Stuart Stanfield from 2002). Without this as a starting point I would not have gotten as far as I have. This is my first attempt at reverse engineering a binary file format, and that dev log is a good introduction for getting into the right mindset.

Tachyon stores assets in a single ~400MB pff archive. The PFF format is something common across NovaLogic games, shared by TTF and (the early) Delta Forces. It’s effectively a tar file- contains a set of records and the unencoded/uncompressed file data in a single archive. (apparently later versions of PFF support compression of data, but that’s not something we need to worry about).

The models in the PFF archive are stored as PAKs (for small things / single ships) and OCFs (for stations, capital ships, other). The PAKs contain model data, image data, and minor rigging information. The OCFs combine a set of PAKs in a tree format, basically like a standard hierarchical skeleton.

What to Expect

The next part in the series will be looking at the individual file formats. Really most of what I’m looking to do with this series is document the format and process so anyone else can do the same. I hate it when information is lost because it went unshared, especially in this day and age- there are quite a few tools out there, people have clearly spent time on this, but very little source code makes it out into the wild.

My end goal is to build a model viewer/exporter, to look at all the shiny PAK and OCF assets and export them in a more friendly format. This’ll be in C# as I prefer it for building tooling that calls for a GUI. I’ll post the end result on Github for all to consume, and hopefully not be C&D’d.

A lot of the work is already done (I’ve already built individual PAK and OCF viewers, but they’re a bit rough around the edges, so I’m redoing bits of the UI) (and it’s not yet in GitHub/public), so this is mostly a retrospective, though there are still a lot of unknowns I’m hoping to solve.

Update:

Code is available on GitHub at: https://github.com/codertao/TTFModelViewer . The code includes everything I’ve done up until now, so, it’s semi-complete, but not that clean.

Seven Segment Thermometer

Hey look, a project!

While I’m waiting for everything to align perfectly for my descent into robotics, I’ve built a thermometer for the dining room. Three times. I’ve now gotten it to some form of a stopping point.

Pieces and Parts

This uses:

• 4x Common Anode 7 segment displays (got from Allied Electronics)
• 2x A6276- a 16 bit constant current (sink) LED driver (from somewhere)
• 2x 10k resistors
• 1x DC Boarduino (from Adafruit)- Arduino clone intended for breadboarding use
• 1x Prototyping PCB (from RadioShack)
• 1x TMP36 (also from Adafruit)
• More 22ga solid core hookup wire then I care to remember

The A6276 is pin compatable with the STP16C596, the apparent Arduino LED driver chip of choice. It was acquired in efforts to fix the legendary Proton LED sign- sadly (or fortunately) the chips on the sign were a different form factor, and the A6276’s were donated to my evil causes. The chips work as a constant current sink; when a pin is set high, it effectively opens a current limited channel to ground; when set low the pin is at high impedance.

The Boarduino was originally intended for use in a cable tester due to it’s awesome size (cable tester may be a different post… or just a set of images). Strange things happened with that, and it is now serving as an overpowered thermometer.

Displays of Seven Segments

Technically the displays are eight segments including the decimal. Which works out nicely for being driven from a 16 bit driver. This 4 digit display is the main reason that the thermometer is getting posted, and here’s why: I’m using a protoboard and 4 7-segment displays. There’s no direct/simple way to connect the LED driver’s to the pins of the displays. 32 pins need to be connected for the displays. 32 small wires. 64 solder joints. In a small space. I present: pain.

There’re probably also a great many samples of ‘bad soldering’ in that photo, but I’m ignoring that for now. Why? Because of this:

I’m 90% certain there is a much better way to do this (Possibly a 4 digit seven segment display?). But it works. And it is awesome.

The wiring for this part, other then being tedious and made of pain, isn’t complex. To prevent excess pain, I have the hookup wires always run parallel. What this functionally means is the pin out on the right side of an LED driver chip is reversed on the left side. This requires some special handling in the software, and is not necessarily ideal, but, we tossed out ideal with the 40 hookup wires. Ignoring the reversing bit, each display has it’s pins hooked up in the same order to a given 8 pin side of the driver chip. The rest is basic stuff- serial out on chip one to serial in on chip two; latch, and clock are shared between both chips, chip output (the R-EXT pin on the datasheets) is a 10k resistor to ground, etc.

I would actually like to do this again as an intro to PCB design project. When I can find time to figure out how to use Eagle.

Sensing Temperature

Temperature sensing is straightforward- the TMP36 sensor is supposed to put out a 0V-someV signal based on the temperature it is currently at, following a roughly specific formula (outputting 500mV at 0C, and increasing at 10mV per degree C). Just a simple analog read and conversion. Roughly.

You can see the TMP36 at the bottom of the pic, snuggled next to 0.1uF capacitor as the spec sheet requests. I tried this without the capacitor and tended to get strange readings quickly (within a few minutes). With the capacitor, they tend to be less frequent (that said, no matter how bad the insulation in my house, it was not 4F last night. I refuse to believe). I started taking a median of several readings in order to solve the issue of strange outliers, and it appears to have worked. No ghosts causing 4F readings yet.

One other quick comment though: The TMP36 outputs 10mv/C roughly. Due to the fact that the Arduino is using a 10bit ADC converter over a 5V range, it effectively reads in 5mv increments. Therefore, we can only read the temperature to an accuracy of about 1 degree F. Which is a bit sad, as it makes those last decimals of accuracy meaningless. Alas.

Conclusion

I’ve built a thermometer! With (reusable) seven segment display! For more cash then I care to think about! I’ve had two other variations of this before: outputting current temperature to a set of normal LED’s in binary (fun, simple, slightly confuses relatives, need right LED size to make it nice), and to an LCD (can display high/low/current readings simultaneously, and be read by non techies). A few other variations could be done relatively easily: an indoor/outdoor meter, monitoring multiple rooms, temperature loss in ducts from straight out of AC to last outlet, etc.

This and a few other projects I’ve done fall under the ‘can be done with 3 output pins and maybe an analog read’ category of projects. I’m plotting to start playing around with the ATTiny series of uC’s to get the project down to component parts / decrease the requisite bulk. We’ll see how that goes.

UPDATE:

Small update. I changed the analog temperature sensor to a digital sensor, the DS18B20 (from Sparkfun), using a set of libraries for the sensor (from Miles Burton). The nicest thing about the new sensor is that it has an onboard 12bit ADC, which lets it sense temperature accurately to the last tenth of a degree that I was wanting. It also doesn’t appear to suffer from the same noisy readings I was getting with the TMP36. The only downside is that it takes 3/4’s of a second to get the 12 bit value and send it back to the Arduino. Which is fine for what I am doing, but it doesn’t seem ideal for other applications.

One interesting other bit about the new sensor: it can operate on only two lines; a ground line, and a data line. The data line is supposed to be connected (via a 4.7k resistor) to a voltage source. When the sensor wants to send a 0 to the uC, it connects the data line to ground, and the uC reads 0. When it wants a 1, the data pin becomes high impedance, and the uC reads a 1 coming from the pull up resistor. I assume it has some kind of in-built capacitor to keep it powered through the 0-bits. Maybe? Probably.