One of my colleagues and I ran into a little trouble with Model Binding in ASP.NET MVC 4 yesterday. We wanted a form to post back an IEnumberable of a complex type, let’s call it IEnumberable. Doing this naively didn’t work. After a bit of googling, and experimentation we found out how to get what we wanted, but the answer was a bit quirky: you must use an array/collection index operator inside your Html.Whatever() lambda, otherwise the Html helper doesn’t know that it should put an array index at the front of the field names in your form. If it doesn’t put this index (in the form of “[0].”, “[1].”), then the MVC model binder can’t work out how to group your fields into instances of a complex type.

Our initial view looked something like this:

@model IEnumerable<ComplexItem>

@{
    ViewBag.Title = "ComplexList";
}

@using (Html.BeginForm())
{
    <h2>ComplexList</h2>
    foreach(var item in Model)
    {
        @Html.HiddenFor(m => item.Id)
        @Html.TextBoxFor(m => item.Title)
        @Html.TextBoxFor(m => item.Price)
    }
    <button type="submit">Submit</button>
}

And the action method it was posting back to looked something like:

[HttpPost]
public ActionResult Index(IEnumerable model)
{
    return View(model);
}

I went away to a meeting and came back to find that the problem had been solved, based on this link. Essentially, you can induce MVC to spit out a prefix on the field names (in the form of “[0].”, “[1].”, etc), which allows the model binder to group the fields together into a collection of complex types. Now, I still felt like we had unfinished business because I wanted to know how it works, so I conducted an experiment.

Referencing Phil Haack’s example, my example becomes something like this:

@model IList<ComplexItem>

@{
    ViewBag.Title = "ComplexList";
}

@using (Html.BeginForm())
{
    <h2>ComplexList</h2>
    for (int i = 0; i < 0; Model.Count; i++) 
    { 
        @Html.HiddenFor(m => Model[i].Id)
        @Html.TextBoxFor(m => Model[i].Title)
        @Html.TextBoxFor(m => Model[i].Price)
    }
    <button type="submit">Submit</button>
}

And I also changed the action method to use IList instead of IEnumberable. This works, but I was trying to determine how it was emitting those prefixes (or at least what was causing it).

So, enter the scientific method:

Hypothesis 1: The collection must be a type that has an “IndexOf” or “FindIndex” method (hence Arrays and IList<T> working, and IEnumerable<T> not working).

To try this I went down the foreach route (ie the first example), but with IList<ComplexType>. No joy. To further test this I used an array as well, still no joy. Hypothesis 1 disproved.

Hypothesis 2: It is the act of calling the [] operator inside the lambda.

So, working backwards from the working code, I normalized out the “Model[i]” bit:

@model IList<ComplexItem>

@{
    ViewBag.Title = "ComplexList";
}

@using (Html.BeginForm())
{
    <h2>ComplexList</h2>
    for (int i = 0; i &lt; Model.Count; i++) 
    { 
        var item = Mode[i];        
        @Html.HiddenFor(m => item.Id)
        @Html.TextBoxFor(m => item.Title)
        @Html.TextBoxFor(m => item.Price)
    }
    <button type="submit">Submit</button>
}

This did not work, which leads me to conclude that it has to do with the act of calling the [] operator in the lambda. Hypothesis 2 confirmed, for now.

I still don’t know how it works, but at least I know exactly what to do to get it to work. This also leaves me in awe of the power of lambdas, and the ability to break a lambda down into an expression tree, and deal with the result in wildly different ways, based on the form of the expression.

UPDATE: I’ve found a feature of Lambdas that would facilitate the behavior we’re seeing here, Expression Trees. You’ll notice that the parameter for Html.TextBoxFor() etc is Expression<Func<…>>. By wrapping the lambda in an expression object, you get the ability to break it down into its components. When you do that you can detect things like whether there is an array index happening etc. I’ll show you what I mean below:

using System;
using System.Linq.Expressions;

namespace ExpressionTrees
{
    class Program
    {
        static void Main(string[] args)
        {
            var p = new Program();
            var data = new string[] { "one", "two", "three" };

            for (int i = 0; i < data.Length; i++)
            {
                p.Investigate(() => data[i].ToLower());
            }
        }

        public void Investigate(Expression<Action> exp)
        {
            Console.WriteLine(exp.CanReduce);
        }
    } 
}

Stick a breakpoint on the “Console.WriteLine” line in Visual Studio, and then take a look at the magic!

The debug string for my expression

As you can see, there’s an object, and it knows that there’s a closure, of which the “data” object is a member, and another closure, of which the “i” array index is a member, and it seems to know the sequence of operations. Let’s dig further:

Inside the Expression

We can see that at the top level we’ve got a NodeType of Lambda, which has a Body. That body is a Call of Method on an Object. Looking at the object:

How you detect an Array Access

We can see that the Object is an ArrayIndex. You’ve got a Left which is the array ((ExpressionTrees.Program+<>c__DisplayClass0).data), and you’ve got a Right which is the index ((ExpressionTrees.Program+<>c__DisplayClass2).i).

Short of decompiling, or finding the source for Html.TextBoxFor, I would think they are analyzing the expression, much like this, and selecting a codepath to handle the result. The graphics geek in me has his propeller hat spinning really fast right now, because you could do some really interesting stuff with this for scene querying.

Buffer Overrun / Overflow

First, the whole buffer overrun thing: it seems that Microsoft have deprecated large portions of the C Runtime on windows (fopen, strcpy, pretty much anything that takes a potentially unsanitary char* as an argument, and writes the data contained therein to another unsanitary char* argument), in favour of the ‘safe’ variants of these functions (which only exist on windows). The gist here is that you need to tell it how long the destination buffer is, so that the function will not overrun the buffer. I can understand why they did this, after the whole security shake-up they had during the Vista development cycle (rumoured to be the reason it took so long). Incidentally, we might see Apple making similar moves after this little hiring.

Apparently these warnings have existed since Visual Studio 2005, but with Windows Store apps, the warnings about these functions have become errors. You can turn them off by defining _CRT_SECURE_NO_WARNINGS, but I’m not sure if I want to. On the one hand, it does concretely close a potential attack vector in any code I write. On the other hand, Microsoft’s series of _s functions (fopen_s, strcpy_s, wcstombs_s, etc) are not part of Standard C, and are thus not portable either. Some people have suggested that if one is re-writing C code to use these functions, they should just re-write it in C++ and use the iostream classes instead, which is great, except most OSS newbies like me don’t have the clout to get a C++ re-write of zlib accepted. So, I’m left with two options: write code that Microsoft have deemed they don’t want running on their OS, or write ugly, macro-heavy wrappers so that the code uses fopen_s on windows, and fopen elsewhere. I’m going to try the second option for now, and see how far that gets me. Don’t be surprised if I blog in the future about how ssimp core team won’t let me commit my code.

Windows Store App Security Model

The next issue that I’ve encountered is more interesting, and it has risen out of the unique space that Windows RT fills in the market, as more than a tablet, but a little bit less than a full-blown Windows PC. If we look at the iPad/iPhone, and Windows Phones as well (I’m, not even going to discuss Android, it’s just wrong to put Linux on a phone), each app is a completely walled garden, except for a few API’s which give access to things like, Pictures, Emails, Contacts – your personal data that apps can add value to. The thing about accessing this data is, as I mentioned before, it is done through and API, you don’t just go poking around the file system opening files as you please. Furthermore, when these apps do try to access the file system using direct, fairly low-level calls, like fopen, they can only see within their own little sandbox. This is fine on a device whose paradigm is that Apps only work on their own data, but Windows is a bit different. Windows is traditionally a desktop OS, I have a Documents Library, and a Pictures Library, and a Downloads Library, and my files are a thing in themselves which transcend my current set of Apps. So, you can access the file system, but it has to be through an API (StoreageFile) if it isn’t within your sandbox, and you can’t even ask for files outside your sandbox unless the paths have come from a FilePicker. I think this model is sound – an app can’t touch anything it didn’t create without your very explicit permission. My goal, however, is to write a Windows Store app version of Assimp Viewer, and so I have a need to use fopen on files outside my sandbox, which was a bit of a conundrum. Luckily though, I’m not blazing a trail here, and I found this post which details a neat strategy which lets you use the StoreageFile API to bring the file into (and out of I presume) your sandbox, so that you can use your low-level C code on it. This shouldn’t be too hard for me to get into the app, because AssimpView is already a Windows only application.

So, my current status is that I’m testing out my build of zlib for a Windows Store App, and it seems to work thus far. I think building the assimp library itself might not be so complicated, because, today’s points aside, any ANSI C code should “just work” when compiled as a Windows Store App static (or dynamic) library.

Digging further, I have found that at present, the zlib functionality is only called by the ‘unzip’ library that also comes with Assimp, which itself is currently only used by the Q3BSPZipArchive class. So, I think I might put in an alternate code-path in either the unzip code, or perhaps even the Q3BSPZipArchive to use the Windows Runtime code. I suspect that modifying unzip.c would be the best approach, because then anyone who writes a custom importer using the Q3BSP one as a template will be able to use my functionality.

It’s settled then, I’ll make an alternate unzip.c (or pre-compiler path therein) which uses the native Windows Runtime API to handle the unzipping. Not tonight though.

The next thing I discovered that the function object classes in the STL (std::plus, std::minus etc), were causing a bunch of errors in Vertex.h, the fix was to find AssimpPCH.h in the “Header Files” folder, and add an “#include <functional> to the list of STL includes.

Rebuilding from there gives me a a dirty ole’ LNK2019 in the assimp_viewer project. I think the CMAKE files aren’t adding a -Ld3d9 directive, so I added an entry to “Additional Dependencies” under link settings to “C:\Program Files (x86)\Microsoft DirectX SDK (June 2010)\Lib\x86\d3d9.lib”

Make sure you set a value for CMAKE_INSTALL_PREFIX, and build the “INSTALL” project. You’ll then need to add the CMAKE_INSTALL_PREFIX\lib folder to your path so that you can run the assimp_viewD.exe file.

So far, so good. My next step is to rip out the D3D9 stuff and upgrade it with D3D11 stuff, so that I can build it as a Desktop / Windows Pro assembly. After that I can work out how to make it into a Windows Store app/library, which will be more challenging because of the requirements of the WinRT platform (eg file access must be asynchronous).

See you next time.

1. They don’t currently support VS2012, but CMAKE should fix this
2. I have to do a “Boost Free” build, so I’ll have to see how that goes.
3. The library uses D3DX, which is deprecated from Windows 8 up, so I’ll have to build it agains the old DirectX SDK first, and then start there.

Anyhow, I’m downloading the DirectX SDK now, so I’ll have to wait. More as it happens.

“With apparent wind-speed of 15 knots, coming from 120° to port, what is my best sail choice, and what speed should I be doing.”

The most important thing here is that you now have a target for the crew to sail for – you know that if you’re not within, say, 10% of your target speed something is wrong and you need to work on your trim, or you’ve got something dangling in the water, or stuck on your keel. Of course, as yachties we often think we have a gut-feel for these sorts of things, but a quantitative answer to the question above is very useful.

In the past I’ve sailed on yachts where the on-board instruments are connected to a laptop, and the it logs the data from the instruments, and presents all sorts of fancy graphs, and some of the more expensive software will certainly give you an answer to questions like the one I asked earlier. However, laptops and water (particularly salt water) don’t mix well at all, and they are not cheap to replace. What’s more, it will take the average weekend warrior an entire season, if not two, to get enough data to start producing useful answers to our questions, and having a laptop on board for all that time is just asking for it to get wet.

So, I’ve been thinking of alternate solutions to this problem and have decided to build a “little-black-box” for yachts. Somewhat like a flight data recorder for aircraft. The key difference being that it needs some sort of interface to capture changes in parameters, like which sails are up, and what the positions of various trimming devices are (such as jib cars).

The first experiment

Anyhow, I’ve made a very basic start on the system by using an Arduino prototyping board to read the NMEA 0183 data from my handheld GPS, and relay that data out through the serial port to my computer.

I plan on building a much more sophisticated bread-board prototype in the near future (have to buy a few more parts from Adafruit), and then I’ll look at stuffing it all into some water-tight packaging and taking it out sailing.

The code

My Arduino “Sketch” (the program which runs on the board) uses Maarten Lamers’ NMEA library which needed a couple of modifications to work with the recent versions of the Arduino API:

In nmea.h, change the reference to WConstants.h to arduino.h:

//#include "WConstants.h"
#include "arduino.h"

In nmea.cpp, change the reference to WProgram.h to arduino.h:

//#include "WProgram.h"
#include "arduino.h"

The code for the actual sketch is very simple at this point, and is really just a matter of reading from one serial port, and writing into another, whilst funnelling the data through the NMEA library:

// -- This borrows very heavily from one of Maarten Lamers' 
// examples on how to use his NMEA library.
#include <SoftwareSerial.h>
#include <nmea.h>
SoftwareSerial nmeaSourceA(13, 12);
NMEA nmeaDecoder(ALL);
void setup()
{
    // Set up serial connection to computer. Eventually we'll write
    // to an SD card instead 
    Serial.begin(57600);
    Serial.println("NMEA Logger... v0.1");

    // Set up serial connection to NMEA source.
    nmeaSourceA.begin(4800);
}

void loop()
{
    // If there's anything to read from the serial port
    if (nmeaSourceA.available())
    {
       // We pass the data into the nmea decoder by calling
       // decode(). This function accumulates the bytes until
       // a valid NMEA sentence has been read. If a sentence has 
       // been read it returns true.
       if (nmeaDecoder.decode(nmeaSourceA.read()))
       {
          // We then write out the sentence
          Serial.print("Sentence = ");
          Serial.print(nmeaDecoder.sentence());

          // The data type term
          Serial.print(" Datatype = ");
          Serial.print(nmeaDecoder.term(0));

          // And the number of terms
          Serial.print(" COUNT(Terms) = ");
          Serial.println(nmeaDecoder.terms());
       }
   }
}

That’s all for today. I’ll keep you posted on how the device develops over time.

Normally, I would use the “tf undo” command at the Visual Studio Command Prompt to sort this out, but yesterday I ran into an odd situation where I could not see the pending changes via the tf command, even though I could see them in Visual Studio. Anyhow, I accidentally found a menu item that lets me find pending changes on any part of the source tree, and gives me the opportunity to undo other users’ checkouts on the server.

The “Find In Source Control” menu item can be found in “Source Control Explorer”.

Fire up Source Control Explorer, and select an item in the main pane. Right-click it and you’ll find the “Find in Source Control” item. This has to sub-items but they take you to the same dialog, with a different set of options ticked.

The “Find In Source Control” dialog, as shown when you select “Status” from the “Find In Source Control” menu.

After clicking “Find” you’ll be presented with the results of a source control query (much like searching through TFS, or comparing). You’ll notice up the top there’s a convenient “Undo” button.

Notice the handy “Undo” button.

This will make it a lot easier to undo orphaned changes for people.

I went through the first couple of experiments but decided to go off the range a little with the second one. Instead of writing a whole bunch of code which goes through and sets individual pins, I decided to try using the Port Registers to set a whole bunch of pins at once. So, what I decided to do was make an 8-bit binary display. I had to use some Bit Math to take the lower 6 bits of the number and put them onto one port register, and the upper two onto the other port register. Thanks to Cosine Kitty for the info on Port Registers.

Here’s a video of my first pass at the experiment, with scrolling LEDs.

And here is the binary counter.

My next step will be to read bytes from the serial port and show them on the 8-bit display.

That’s all folks.

1. No item should ever go “un-processed” – if it fails to go down the normal handling path, it should go down an error handling path – either way it should leave the main queue.
2. There must always be a mechanism for a human to intervene, and correct (or choose to ignore) malformed data; this makes for nice error-handling path for point 1.