Techquila Shots

Its a pleasure to announce the first release of Metatribble, an attempt to implement some of the concepts of semantic annotation that I talked about in a previous post. Metatribble is currently packaged as a Ubiquity command and right now doesn’t do an awful lot except for mark up interesting entities with RDFa, but even just that is kinda cool :-)

To this stage the project has been a collaboration between myself and Inigo Surguy, but making use of the tremendous work done by Jeni Tennison on the rdfquery plugin for jQuery - we are truely standing on the shoulders of giants here. We would welcome any comments/suggestions/offers of help.

Links to the project and its source code and the installer for Ubiquity can all be found on the Metatribble project homepage.

BTW We are designating this release a zeta because the concept of “beta” has been somewhat devalued by beta-ware such as gmail, we feel we need to wrap around the alphabet.

Last night’s Oxford SWiG meeting was interesting and sociable as usual. There were three great presentations - Jeni Tennison on rdfQuery, a jQuery-like Javascript library for parsing, querying and generating RDFa markup; Iain Emsley presented a WP plug-in that creates RDF graphs for blog posts showing a nice use of multiple ontologies; Laurian Gridinoc talked about the plans for PowerMagpie - with lots of ideas for navigation/presentation of large taxonomies and complex ontologies.

As usual though the real action was in the pub. One of the things we got to discussing was whether rdfQuery could be used to create stand-off markup on someone else’s content. Inigo Surguy pointed out that using tools such as Greasemonkey it should be pretty easy to get rdfQuery to scrape a page for the RDFa it contains and to add custom scripts to do something cool with that data. The problem comes when trying to persist any new RDF statements you might create. RDFa is a syntax for embedding RDF within HTML - so if you are in control of the page that you are adding the markup to, it is trivial to persist that markup simply by saving the modified file. If you are not in control of the page then you have some problems. The easy case is when the publisher of the page has already identified things that you might want to talk about and wrapped them in some RDFa. In this case you can simply add some more statements about those entities. What is harder is if the publisher of the page hasn’t marked up anything with RDFa. What is needed is a “bootstrap” mechanism to locate entities that you might want to talk about.

That is where OpenCalais comes in. The OpenCalais service takes content and locates entities within it, returning the content with markup added that identifies the entities within. Using some custom code interfacing to rdfQuery, it should be possible to turn the results from OpenCalais into RDFa, then you can do all the funky stuff you want with the RDF and serialize it to some persistent store (either on another web service such as the Talis platform or maybe to a local persistence mechanism such as Gears). Now, when you return to the page, your script again goes to OpenCalais to get the entities identified within and again turns this into RDF, but now you can smoosh in the RDF from your persistent store to retrieve all that cool markup you added.

What’s even better is that because OpenCalais has unique identifiers for the entities it recognizes, if you then visit another page that contains a reference to the same entity you should be able to pull in your extra markup automatically. I’m pretty sure that with this approach it should be possible to build up a personal knowledge store that can be merged into web pages as you view them, combine with some clever javascript to present that information and to allow you to extend the set of statements in the store and you have something really rather cool.

Just need to code it now ;-)

If any of you are using NPCL to write your topic map ontologies, you should know that we have just released a new version of the NPCL Schema Editor. Includes MSBuild support, compiling multiple NPCL files into a single file, auto-generated documentation and much much more. Fun for all the family and competitively priced at $0. For more info see http://www.networkedplanet.com/ontopic/2009/03/npcl_schema_editor_for_visual_1.html

In some deployment scenarios it is not possible to bundle everything into a single installer. This may be because you need to install some third-party software, or because your build process forces you to split a single product across multiple MSIs.

Although there is not much in the WiX documentation about this, it is possible to chain multiple MSI installers in a setup.exe wrapper so that the installers execute one after the other. The key to this is the setupbld.exe command-line tool that comes with the WiX 3.0 installation. Jon Torresdal has written a useful article on using setupbld.exe to force elevated privileges for an installer under Vista, and I recommend starting there for an explanation of how to prepare your build to generate setup.exe files. In addition to doing this, setupbld.exe can combine multiple MSI files into a single setup.exe installer. To do this use a command line like:

setupbld.exe -out $(TargetDir)setup.exe -msu FirstInstaller.msi -msu SecondInstaller.msi -setup $(ProjectDir)setup.exe

At the moment I’m in the middle of updating the NetworkedPlanet NPCL Schema Editor so that it works with Visual Studio 2008, a task that sets new standards for the word ‘frustrating’. Anyway, having got my whine out of the way, I did find the fix for something which no amount of Googling revealed and I’m writing it up here in the hope that it might save some other poor soul from a bruised forehead.

One of the key steps in deploying a Visual Studio package is writing all of the correct package registration details into the target machine’s registry. For my deployment I wanted to use the WiX toolset to create my installer. According to this MSDN article, you can use the tool regpkg.exe in the Visual Studio SDK to generate a .wxs file that contains the necessary WiX magic to get all of the package registration entries written by the installer. Its then just a question of writing the rest of the installer to put files in the right place, include the .wxs file with the right sort of invocation and off you go.

On testing the deployment on a clean Visual Studio installation I found that the project template I had created to allow users to choose to create an NPCL project didn’t show up in the New Project dialog box. I could see that my Project Package was properly registered with Visual Studio on the target machine (as proof of which I could see it displayed in the list of registered extensions in the Help > About dialog box). I could also see that the project template was installed into the Common7\IDE\ProjectTemplates directory as expected.

After much cursing, rebuilding, redeploying and retesting I finally went back to first principles and decided to compare my package registration key by key with another project package extension that I knew worked - in this case the WiX installation. What I found was interesting. This is the code generated for my project package’s template registration by regpgk.exe:


<Registry Root="HKLM" Key="Software\Microsoft\VisualStudio\9.0\NewProjectTemplates\TemplateDirs\{41a84a7e-462c-4a74-9559-3c2568001dc2}\/1">
<Registry Name="SortPriority" Value="100" Type="integer" />
<Registry Name="TemplatesDir" Value="[$ComponentPath]\.\NullPath” Type=”string” />
</Registry>


And here, for comparison is the Wix registration

<RegistryKey Key="NewProjectTemplates\TemplateDirs\$(var.WixVsPackageGuid)" Action="createAndRemoveOnUninstall">
<RegistryKey Key="/1">
<RegistryValue Value="WiX" Type="string" />
<RegistryValue Name="SortPriority" Value="30" Type="integer" />
<RegistryValue Name="TemplatesDir" Value="[VsProjectTemplatesWixDir$(var.VsVersion)]” Type=”string” />
</RegistryKey>
</RegistryKey>


The key difference is that the generated registration doesn’t set the default value for the /1 registry key. It appears that this default key value is actually the label for the project type that is displayed in the New Project dialog box, but the regpkg.exe tool doesn’t generate a <Registry> tag to set its value. In the WiX schema there appear to be multiple ways to do this, and I found the syntax below worked for me:

<Registry Root="HKLM" Key="Software\Microsoft\VisualStudio\9.0\NewProjectTemplates\TemplateDirs\{41a84a7e-462c-4a74-9559-3c2568001dc2}\/1" Type="string" Value="NPCL">
<Registry Name="SortPriority" Value="100" Type="integer" />
<Registry Name="TemplatesDir" Value="[ptNPCL]” Type=”string” />
</Registry>

As an aside, the regpkg.exe generates a .wxs file that uses the schema supported by WiX 2.0. I’m using WiX 3.0 for my installer, and it reports these <Registry> elements as deprecated. However, it does work and I can now get on with finishing off my installer.

I hope that this tip saves some one the hours that I will never get back again ;-)

It is very interesting to see that Amazon have now made available over 1TB of public data. Its great that all of this data is now available in one place, ready shredded into queryable structures that allows developers to get to grips with it and start to do something really interesting. But wait a minute, if I want the DBPedia dump I have to go here…, if I want the Wikipedia english articles I need to go over there. If I want the US census data from 2000 its this place, if its the census data from 1990, its somewhere else. Oh and don’t even get me started on having to choose between Windows and Linux. What these data sets are are essentially separate database snapshots that you can load into your own EC2 instance in the Amazon cloud and then start processing.

…and thats kind of disappointing. Having lots of open data is a great start, but it is only a start. And here is the challenge - there are no consistent semantics acrosss these data sets, there is a great deal of wet-ware time that needs to be invested in working out the linkages between them and in getting hold of some consistent notions of identity that could assist in merging. The easy way out is to pick and choose and make a “mash-up”, but there is nothing reusable in a mash-up, and a million mash-ups do not make a viable platform for building the really cool apps of the next decade on. Topic maps on the other hand has a model for reflecting a consistent notion of identity, for reconciling different identity notions and different entity schemas.

There’s the challenge - can we integrate all of this data using topic maps ? Can we make use of the tools provided by the Amazon platform to build something even more cool - a cloud based index of the entities and relations in these data sets ? Because I believe that when we can do that we will really have 1TB (or given the expansion of the topic maps model probably 1PB ;-) of useful knowledge, rather than 1TB of bits and bytes that you can hack with a mash-up.

There’s the glove. Who’s going to pick it up ?

Definitely an experience to file under the “it just works” category. Upgrading from my old old old installation of MoveableType to Wordpress was almost totally painless. There were only a couple of small “gotchas” that I haven’t quite got around to fixing yet.

Firstly my collaborator on the Pepys topic map stuff, Stuart Brown has lost his credit on his posts. For some reason the importer decided that all posts has been written by Stu and I had a choice of either going through and manually editing each post by hand or just assigning them all to me. Always one to take the easy way out I opted for the latter. Sorry Stu! Still, if he is that upset, I can always let him come on over and do the manual editing, eh ?

Secondly the RSS feed will move so those of you who have a subscription will need to update your readers to look at the feed for posts or for comments (or both!) . I’ll see if I can force some sort of redirect through the magic of .htaccess, but hey what did I just say about always taking the easy way ?

The theme is “Love The Orange” for which my thanks go to the creators at Web Design Creatives.

I really like the editing interface in WP - spell checking and a nice WYSIAWYG HTML editor will mean I have fewer excuses for producing poorly written garbage. The whole admin interface is neat and intuitive and possibly my only gripe so far is that I can’t find a quick way to bulk edit more than the 15 posts you can display on a single page. And WP supports more ways to get a post up than my venerable MT isntallation did. Speaking of which, if anyone has any tips on good tools for posting to WP (from desktop and/or from a Symbian-based phone) please let me know in the comments section!

In the four years since I last did anything with this blog, things have moved on a bit in terms of blogging systems, so I am considering replacing this MT installation with something a bit more hip, with it and down with the kids….well OK with WordPress anyways. As you can see from the page layout my design skills are somewhat poor but as a dedicated geek I hold fast to the belief that there is nothing that can’t be fixed by upgrading.
So all things being equal this weekend I’m going to try and get WordPress on to this site, and transfer all of my old posts over to it. My host has PHP and MySQL so I have the basic WP requirements, so hopefully it should be easy enough. If anyone has any tips or knows of any gotchas in moving from MT 2.6 to the latest WordPress please let me know.

Yes its been a very long time since I last added anything to this blog. Nearly 4 years in fact! I’ve not been on a round-the-world tour or used the last four years to perfect my violin technique or been living in a closed community that uses only 19th century technology (aka Windows 3.1). Most of my time has been spent working on stuff at my company NetworkedPlanet. And running a startup, and keeping up wiht Topic maps and .NET (amongst other things) have basically taken over most of my time.
But now its time for a restart of my blogging career.
Roll on the next post (hopefully without the 4 year gap this time).

The Pepys-Map is a topic map of the people, places and events described in the famous 17th Century diaries of Samuel Pepys.
This round-up topic map covers all of the entries from June 1661 through to March 1662. You can either download the full XTM representation of Pepys-Map or browse the HTML rendition of Pepys-Map online.