The windows API

I'm not in general a fan the win32 api, which for those of you who know me well won't come as much of a surprise. But while working on a the bottom layer of the ACID compliant store stuff for MysteryMachine , I realised that I could just use TxF on versions of windows that are late enough. And indeed it seems stupid not to, so I decided to rewrite just the bottom layer of the transaction handler to use TxF when available.

Now for those who don't know , MysteryMachine is written in Python which is not the most natural language for direct access to the Win32 API which is primarily called from C/C++. There are some bindings also for C#, but I get the impression there is quite a lot between C# and the kernel when used this way.

So it became time for me to see how I could call this functions easily from Python. I looked into the Python toolbox and found ctypes which is perfect for the job. You can literally just mention a DLL by name, and construct a tuple of arguments. The arguments have to be made from special ctype objects which represent the types primitives available in C. You can read the documentation and the link above to find out more about ctypes itself.

Now I just wanted a simple wrapper around the TxF functions so I didn't have to go through all that rigmorole of type conversion each time, so it seemed that each function I wrote would be of the form of:-

def Windows32Function(*args,**kwargs):

type_list [ HANDLE, LONG , LONG, POINTER, LONG ,LONG, HANDLE, LPCWSTR ]
    out_args = []
    for arg,argtype in zip(args,type_list):
         out_args.append(argtype(arg))
  
    ctypes.dll.Windows32Function(*out_args)

Now the think I really don't like about the Windows Api is the large number of arguments that windows functions regularly take , and which 99% of the time you can just you the same default option, and it struck me - Python has first class support for keyword arguments, that what the kwargs argument above is for.

So I came up with the following:-

def do_args(args,kwargs,name,dflt,factory = None):
    if args:
        dflt = args.pop(0)
    val = kwargs.get(name,dflt)
    if factory:
 val = factory(val)

    return val


def CreateFileTransacted(*args, **kwargs):
    args = list(args)
    lpFilename = do_args(args,kwargs,'filename',None,LPCTSTR)
    dwDesiredAccess = do_args(args,kwargs,'desired_access',const.GENERIC_READ,DWORD)
    dwShareMode = do_args(args,kwargs,'share_mode',0,DWORD)
......

Looking at the code above , I can use the do_args function to take the grunt work out of processing each argument. , You should be able  see that do_args takes an keyword name, a default value and a type in that order, as well as the arguments lists from the original function.
You might be wondering about the first line of the CreateFileTransacted wrapper, it is important that we use a args in a list forms as the tuple we start with (as all tuples in python are) is immutable so we can't pop the arguments from it as we go.

It takes an argument out of the positional list, check the args name in the keyword list, (keyword overrides positional in this implementation) and if neither is there will use the provided default. Then finally it casts it into
the correct type for ctypes to use. Here I name the arguments before I call the DLL entrypoint but thats not necessary a list can be built up like the original version did.

What's really good about this is that do_args handle all of the marshalling ctype requires the use to do, so is quick to write and the code is simple enough to be easy to verify. But the really amazing thing about this is it makes the windows API much more pleasant to work with. I managed to get most of the TxF implementation included all the windows functions done in mainly a day. There has been lots of messing around since to check it multiple platforms and deal with the inconsistencies but it was much quick to code windows calls this way. There are two big contributors to this ease which I don't normally get with windows programming and the first was the simple interactive nature of Python , I could very simply load the wrapper modules into a python console session and start calling the  Windows API directly.

Normally this would be a pain in the proverbial , as the number of arguments is a large and would each have to be specified in the right order and with the right number of intervening NULL arguments for the features unused, but because I could really on the wrapper providing me with all the sensible defaults I only needed to provide the one or two arguments I was actually using it became much quicker to find out exactly what the real behaviour of windows was. I actually found it felt like windows behaved for once - I was quite shocked.

However as I was to discover while TxF behaved fine, windows file semantics meant that I had more problems head, but I save that for a later post.