Porting Open Asset Import Library (Assimp) to WinRT (4)

So, I changed my mind about the whole zlib vs System.IO.Compression, partly because the latter is not already a Windows Runtime accessible type (afaik), and secondly because the code would not be at all portable, and I’d like to be able to contribute my work back to the community if I can. I’ve run into a couple of other problems though, one to do with Microsoft’s war on Buffer Overrun exploits (fair enough, Windows XP was a real problem that way), and the other to do with the security constraints that come with Windows Store apps.

Buffer Overrun / Overflow

First, the whole buffer overrun thing: it seems that Microsoft have deprecated large portions of the C Runtime on windows (fopen, strcpy, pretty much anything that takes a potentially unsanitary char* as an argument, and writes the data contained therein to another unsanitary char* argument), in favour of the ‘safe’ variants of these functions (which only exist on windows). The gist here is that you need to tell it how long the destination buffer is, so that the function will not overrun the buffer. I can understand why they did this, after the whole security shake-up they had during the Vista development cycle (rumoured to be the reason it took so long). Incidentally, we might see Apple making similar moves after this little hiring.

Apparently these warnings have existed since Visual Studio 2005, but with Windows Store apps, the warnings about these functions have become errors. You can turn them off by defining _CRT_SECURE_NO_WARNINGS, but I’m not sure if I want to. On the one hand, it does concretely close a potential attack vector in any code I write. On the other hand, Microsoft’s series of _s functions (fopen_s, strcpy_s, wcstombs_s, etc) are not part of Standard C, and are thus not portable either. Some people have suggested that if one is re-writing C code to use these functions, they should just re-write it in C++ and use the iostream classes instead, which is great, except most OSS newbies like me don’t have the clout to get a C++ re-write of zlib accepted. So, I’m left with two options: write code that Microsoft have deemed they don’t want running on their OS, or write ugly, macro-heavy wrappers so that the code uses fopen_s on windows, and fopen elsewhere. I’m going to try the second option for now, and see how far that gets me. Don’t be surprised if I blog in the future about how ssimp core team won’t let me commit my code.

Windows Store App Security Model

The next issue that I’ve encountered is more interesting, and it has risen out of the unique space that Windows RT fills in the market, as more than a tablet, but a little bit less than a full-blown Windows PC. If we look at the iPad/iPhone, and Windows Phones as well (I’m, not even going to discuss Android, it’s just wrong to put Linux on a phone), each app is a completely walled garden, except for a few API’s which give access to things like, Pictures, Emails, Contacts – your personal data that apps can add value to. The thing about accessing this data is, as I mentioned before, it is done through and API, you don’t just go poking around the file system opening files as you please. Furthermore, when these apps do try to access the file system using direct, fairly low-level calls, like fopen, they can only see within their own little sandbox. This is fine on a device whose paradigm is that Apps only work on their own data, but Windows is a bit different. Windows is traditionally a desktop OS, I have a Documents Library, and a Pictures Library, and a Downloads Library, and my files are a thing in themselves which transcend my current set of Apps. So, you can access the file system, but it has to be through an API (StoreageFile) if it isn’t within your sandbox, and you can’t even ask for files outside your sandbox unless the paths have come from a FilePicker. I think this model is sound – an app can’t touch anything it didn’t create without your very explicit permission. My goal, however, is to write a Windows Store app version of Assimp Viewer, and so I have a need to use fopen on files outside my sandbox, which was a bit of a conundrum. Luckily though, I’m not blazing a trail here, and I found this post which details a neat strategy which lets you use the StoreageFile API to bring the file into (and out of I presume) your sandbox, so that you can use your low-level C code on it. This shouldn’t be too hard for me to get into the app, because AssimpView is already a Windows only application.

So, my current status is that I’m testing out my build of zlib for a Windows Store App, and it seems to work thus far. I think building the assimp library itself might not be so complicated, because, today’s points aside, any ANSI C code should “just work” when compiled as a Windows Store App static (or dynamic) library.

Porting Open Asset Import Library (Assimp) to WinRT (3)

So I’ve been doing a bit of digging about in the Assimp source, and it seems that, for the core library at least, the only dependency is on zlib – which it builds in your solution on Windows. My next possible step, then, could be to move the code from this zlib project over to a Windows Runtime Component project. However, I’m a little reticent to do this, since the System.IO.Compression namespace can do all the compression stuff, and to an extent it depends on whether it is easier to port zlib, or modify Assimp so that it doesn’t use zlib. My hope is that I can keep my changes to Assimp isolated to a fairly low level in the codebase, so that it is reasonably easy to keep in harmony with their continuing efforts, and porting zlib seems more likely to achieve that goal.

Digging further, I have found that at present, the zlib functionality is only called by the ‘unzip’ library that also comes with Assimp, which itself is currently only used by the Q3BSPZipArchive class. So, I think I might put in an alternate code-path in either the unzip code, or perhaps even the Q3BSPZipArchive to use the Windows Runtime code. I suspect that modifying unzip.c would be the best approach, because then anyone who writes a custom importer using the Q3BSP one as a template will be able to use my functionality.

It’s settled then, I’ll make an alternate unzip.c (or pre-compiler path therein) which uses the native Windows Runtime API to handle the unzipping. Not tonight though.


Porting Open Asset Import Library (Assimp) to WinRT (2)

Last time I got to the point where I had found that I can’t build the code in Windows 8 because DirectX 11.1 has deprecated a bunch of DirectX 9 functionality, so I installed the Jun 2010 DirectX SDK to get up and running. My plan is to get it building as a Windows 8 Desktop library, and then rip out the code that won’t e compatible with WinRT.

The next thing I discovered that the function object classes in the STL (std::plus, std::minus etc), were causing a bunch of errors in Vertex.h, the fix was to find AssimpPCH.h in the “Header Files” folder, and add an “#include <functional> to the list of STL includes.

Rebuilding from there gives me a a dirty ole’ LNK2019 in the assimp_viewer project. I think the CMAKE files aren’t adding a -Ld3d9 directive, so I added an entry to “Additional Dependencies” under link settings to “C:\Program Files (x86)\Microsoft DirectX SDK (June 2010)\Lib\x86\d3d9.lib”

Make sure you set a value for CMAKE_INSTALL_PREFIX, and build the “INSTALL” project. You’ll then need to add the CMAKE_INSTALL_PREFIX\lib folder to your path so that you can run the assimp_viewD.exe file.

So far, so good. My next step is to rip out the D3D9 stuff and upgrade it with D3D11 stuff, so that I can build it as a Desktop / Windows Pro assembly. After that I can work out how to make it into a Windows Store app/library, which will be more challenging because of the requirements of the WinRT platform (eg file access must be asynchronous).

See you next time.

Porting Open Asset Import Library (Assimp) to WinRT

With the advent of Microsoft’s Windows 8, and it’s ARM-compatible Windows RT variant, Microsoft have really jumped headlong into the mobile device arena. I think the fact that developers will now be able to develop a single code-base that works on Phones, Tablets, Laptops and Desktop computers, will be quite a big deal for their platform, but it comes with one particularly difficult challenge: the code must only be compiled against what is know as the WinRT stack. This is a subset of the full Windows 8 SDK which is common to all hardware platforms, and it presents a challenge because none of the libraries we’re used to using right now (libpng, libjpg, etc, etc, name your library here). So, I’ve decided, mainly for interest’s sake, to try and port the wonderful Open Asset Import library, commonly referred to as Assimp, to the WinRT platform. Now, I probably would use this in a toolchain more than a runtime engine, but it is still a good test case. I started on this project today, and I can see a few troubles ahead of me:

  1. They don’t currently support VS2012, but CMAKE should fix this
  2. I have to do a “Boost Free” build, so I’ll have to see how that goes.
  3. The library uses D3DX, which is deprecated from Windows 8 up, so I’ll have to build it agains the old DirectX SDK first, and then start there.

Anyhow, I’m downloading the DirectX SDK now, so I’ll have to wait. More as it happens.