Excellent article! I think it will become required reading for tool writers and their clients and managers. This article will make the game development world a better place!

(I bet you'll get job offers as a result of this article.)

Hi Gregg,

Nice article. I felt that everything seemed obvious at first glance, but it is the kind of things we don't necessary do, often due to lack of time given to these tools. As you pointed out, they are often made by a programmer as a first quick test to work with its data/engine, and not engineered enough toward real production usage.

I have a question about automating "one-step build process". What's your whole data pipeline strategy to do that?

For a current project, I wrote scripts checking date-based dependencies (ala Makefile) between source data on a server and locally converted data, and running commands to export everything that has changed. Theorically this seemed ok at first, but:
- As the project gets bigger, checking all files dates is getting longer and longer, annoyingly reaching several minutes. We have a manual way of selecting files to convert (drag & drop file to an icon), but the problem still apply for other people who need your latest data.
- We had plenty of problems due to non-synchronized computers (local PC has 2 minutes more than server. When new data is saved on server it uses server date, and thus comparaison with local dates are incorrect).

Both problems were introduced at first because of the separation between server based source data and local output data. I originally did that because our target platform require embedding data on each compilation so it was much faster to grab them locally each time.

I can probably find a solution (storing output data on the server to avoid date problems and the need to global output, then automatically copying an archive locally for programmers).

But I'd be interested in hearing about your strategy to handle automatic conversion of data.

Thanks.