Jack Dermody

Winter Blues Good for Ask Bluey

2011-10-11T07:51:00.001+11:00

It's been a long winter here in Sydney. Just as spring seemed certain to arrive, winter instead resumed with a vengeance - biting southerly winds, temperatures down to (gasp) 7 at night, and otherwise all round misery. It's back to Ugg boots and heavy winter doonas and hot chai lattes. Even Masterchef is back on TV – that unfortunate dreary bastion of cold winter nights.

However it’s also meant that I have had plenty of time for Ask Bluey and have been making good progress on getting the second beta version out the door. This version has been about polish, simplification and stability and I think these goals have been met.

If you haven’t visited in a while then feel free to take a look – after all, who doesn’t want their own private search engine, scouring the internet 24/7 for personally relevant pages?

Data Storage, Caching and Retrieval in Azure

2011-08-05T09:55:00.001+10:00

Both web and worker roles have a variety of places to store data in the Azure cloudscape. Since processing large amounts of data is likely the raison d’etre of cloud applications in the first place, understanding all the possible places that data might be stored (and their trade-offs) is of some benefit when designing cloud applications.

What follows is more of my personal brain dump based on my experience with Azure. I don’t include performance metrics because I suppose them to be continually changing and subject to variability based on the application in question. I recommend collecting performance metrics in the design phase to give further weight to your own design choices.

LOCAL Memory (per Instance)

Obviously the fastest place to cache your data, but sadly quite limited in size (depending on the size of the Azure instances that you choose (and pay for)).

Also sadly not shareable between multiple instances running in the same web role. Storing session state on one machine helps you very little when the next request lands on the next machine in your web farm.

LOCAL STORAGE (PER INSTANCE)

Storing to disk has exactly the same benefits and trade-offs that one might expect – a far greater capacity than local memory but far longer read and write speeds. Where possible, asynchronous writes may be possible to ameliorate some of the performance penalty. Again, not shareable between instances.

SQL Azure

Since web roles with multiple instances often do need a place to share session data, the database represents a logical place. The ACID properties of a relational database mean that even when individual instances of your application die and are recycled, the user should see no difference in their session from the remaining instances that are still serving requests.

However the size constraints of SQL Azure databases (and expense involved) mean that not every piece of data should simply be bundled into the database. Large blobs of data are certainly a good candidate for blob storage, but their timestamps and other meta information might be stored in the database to aid retrieval time and to integrate with stored procedure logic.

Blob Storage

Blob storage is the cheapest way to store large amounts of data “in the cloud”, although the current offering is not especially compelling, performance wise. Since you pay per transaction it also represents a poor choice to put things that you will be frequently accessing.

Blob storage is like the “stack” at your local library – non frequently accessed things can be stored at low cost for large amounts of time, and brought into the forward caches as appropriate.

Windows Azure AppFabric Caching

An alternative place to share session state between instances, AppFabric caching sounds good in theory but suffers a little in practise. The idea is that it represents a high speed storage location that is easily shared between instances. The reality is that it will shut you off if you use it too much, and the performance does not seem especially compelling. It is however, considerably faster than blob storage.

Putting it all together

A well designed Azure application will take into account all the strengths and weaknesses of the storage options available and work with each accordingly.

Often data seems to move like a waterfall across the different tiers. If an application might require some particular data it might check first with a centralised and reliable store such as the SQL database as to where that data is located and its latest timestamp. Then it might check its various caches for the blob’s handle and timestamp – first in local memory, then local storage, then the AppFabric cache and finally from blob storage. When it finds its data it might refresh the other (empty or invalidated) caches that failed it along the way.

Other times a backend process might invalidate some piece of non-SQL data, update both blob and cache storages and then send a message to each role instance that the data has changed. Each role instance in turn can then query the cache service for the data, and fall back on blob storage if the cache drops the ball. The role instance might store that data in local memory and local storage and simply assume that the data remains valid until a notification to the contrary is received.

But as with all advice, your mileage will vary ; )

Getting the Redirected Url from a WebClient

2011-08-05T08:46:00.001+10:00

WebClient in .NET is a handy class for network access, but sometimes you need things that it doesn’t expose “out of the box”. Fortunately it’s quite simple to extend.

One of the handy features of the class is that it automatically handles redirects on your behalf. Suppose that you make a request to some URI that returns an HTTP status code of 301 – if the AllowAutoRedirect property on WebClient is set, it will automatically parse the HTTP response and make a subsequent call to the redirected URL all by itself. Nice.

In my case however I wanted to find out the actual URL that it had redirected to, so I extended WebClient as below to always save the Uri that it was last getting a response from. This Uri would be different from the request Uri if the server had returned an HTTP status code indicating redirection.

public class MyWebClient : WebClient
{
    Uri _responseUri;

    public Uri ResponseUri
    {
        get { return _responseUri; }
    }

    protected override WebRequest GetWebRequest(Uri address)
    {
        HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
        request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
        request.AllowAutoRedirect = true;
        return request;
    }

    protected override WebResponse GetWebResponse(WebRequest request, IAsyncResult result)
    {
        WebResponse response = base.GetWebResponse(request, result);
        _responseUri = response.ResponseUri;
        return response;
    }

    protected override WebResponse GetWebResponse(WebRequest request)
    {
        WebResponse response = base.GetWebResponse(request);
        _responseUri = response.ResponseUri;
        return response;
    }
}

Azure Table Storage Helper Classes

2011-08-04T09:25:00.001+10:00

Before I discovered that working with Azure’s table storage was not the be all and end all that I had once thought (see previous post), I put together some base classes that implement the key recommendations from the Whitepaper for Programming Table Storage.

These classes make working with Microsoft.WindowsAzure.StorageClient a little easier and less verbose, and means that you can use the following class to define a log table for your Azure cloud application. (You need to create the empty table first with something like Azure Storage Explorer. Note that Azure table names need to be lowercase or strange things might happen!)

The constructor defines the table name “log”, the account credentials to use (here as a static variable ABB, and whether to use https in connecting to the table (in this case false).

The second part of the class definition defines the structure of the table row, including the partition and row keys. In this case the partition key is the type of log message (error, warning, etc) and the row key is the timestamp of the message (in descending order). The message column contains the actual log message.

Note that the partition key and row key are always required for each table and together form the primary key for that row.

internal class LogTable : TableBase<LogTable.TableContext>
{
    public LogTable() : base((baseAddress, credentials) => new TableContext("log", baseAddress, credentials), AccountCredentials.ABB, false) { }

    public class Row : TableServiceEntity
    {
        public Row(string message, LogType type)
        {
            PartitionKey = type.ToString();
            RowKey = (DateTime.MaxValue.Ticks - DateTime.UtcNow.Ticks).ToString();
            Message = message;
        }
        public Row()
        {
        }

        public string Message { get; set; }
    }
    public class TableContext : ContextBase<Row>
    {
        public TableContext(string tableName, string baseAddress, StorageCredentials credentials) : base(tableName, baseAddress, credentials) { }
    }

    public static void AddMessage(string message, LogType type = LogType.Error)
    {
        try {
            var context = new LogTable().Context;
            context.Add(new Row(message, type));
            context.SaveAsync();
        }
        catch {
        }
    }
}

Adding a new log message

LogTable.AddMessage(“some message”);

LogTable.AddMessage(“some message”, LogType.Warning);

Getting a list of all log messages

var logMessages = new LogTable().ReadOnlyContext.Query.Select(r => r.Message);

Download

To The Cloud!

2011-07-30T08:52:00.002+10:00

First of all, what a great job Marketing has done with The Cloud. Nobody knows what the hell it means, but everyone seems motivated to get on the cloud, somehow, anyhow. At the heart of the cloud is a big secret that is easily glossed over – the cloud is nothing new at all, just more of the same - big servers, centralised access, the same ideas that computing has been using from day one (there used to be mainframes that took up an entire office suite, now modern computers live in giant warehouses out in the sticks. That right there is 50 years of Progress).

Aside from the brief and increasingly irrelevant “PC revolution” (the idea that large clusters of small machines would be cheaper than one large mainframe – an idea carefully promulgated by Microsoft and other interested parties), computing has always been a big business kind of thing to do, with big business efficiency gains of centralization and commodification the driving forces behind the push for cloud services.

Anyway, I too partook the cool aid and moved Ask Bluey to the cloud this year, from it’s previous home on shared hosting. Shared hosting was actually surprisingly efficient and cost effective – it was able to accommodate thousands of visitors a day, but it was most definitely grinding to a halt when you threw in web crawlers, which accounted for around half of the traffic to the website).

The premise of being able to scale out Ask Bluey as traffic ebbed and flowed was hard to resist, but the reality is that architecting for the cloud really quite difficult to get right (although the end result is “better”, in the same way that truly cross platform C++ code feels “better” than code with implicit assumptions about bit alignment, or god forbid, non standard compiler extensions).

The first big mistake I made was buying into the NO SQL movement’s claim that the future of databases lay in non relational databases. The idea is that you still have tables to put your structured data into, but that you have to manage the keyed links between these tables yourself (while a SQL database enforces referential integrity on your behalf). What you gain is that these tables can be partitioned across machines, which means that they scale out better. Maybe.

What I didn’t realise is that the “table storage” that the cloud providers offer is not NO SQL, but some weird hybrid of blob storage with two keys rather than one. While it would certainly be possible to use table storage instead of a SQL database, it is not a good idea for two reasons. First of all, the performance is not very good. I found a table based lookup to take twice as long compared to a blob based lookup, which is an order of magnitude slower than a local disk based retrieval. When you have to retrieve from multiple tables and deal with “upsert” semantics, the performance penalties are cumulative and prohibitive.

The second reason to use SQL over table storage is that you have to pay for each transaction. They are billed in blocks of 10,000 per cent, so they sound so cheap that you don’t have to think too hard about them. But they can add up! In one month my “table storage” architecture blew $250 on these seemingly insignificant transactions. Once I had moved back to SQL, the same usage pattern cost $10 per month, as SQL databases are billed on instance size rather than per transaction.

Table storage still has a place in a cloud based architecture, but it only really makes sense in a few limited scenarios. An ideal usage would be an archive of tweets on a per user basis. The partition key is on the user, and the row key is the date. Buy keeping these large wads of data out of the database the SQL instance size is kept down, and the performance penalty becomes less important because of the “archive” aspects. Obviously nobody wants their entire twitter history every single day, but it might be useful to get once or twice a year (the most recent tweets could be in the SQL database).

In this the way table storage becomes ancillary to a good old fashioned SQL database and again nothing has really changed in any aspect of computing, despite the hype. In some ways this is reassuring ; )

First post in a Long Time

2011-07-04T09:45:00.001+10:00

This is the obligatory – wow, it’s been years since I last posted. In the last two (2!) years I have gotten married, become the father of a rather lovely daughter, moved closer to the beach, and launched a large and ambitious project (Ask Bluey).

I feel it’s time to pause, reflect and record - and blogging seems a perfect way to approach that.

So stay posted…

; )

New Article on CodeProject - Search3D

2009-05-19T17:00:00.003+10:00

In this article I talk about how you can use Singular Value Decomposition and Latent Semantic Analysis to cluster documents and to reduce multi-dimensional data down to three dimensions, which is visualisable in a 3D viewer.

Back Button in RIA (Rich Internet Applications)

2009-05-15T11:59:00.003+10:00

There is an inherent problem with the technologies that herald the advance of the internet from static text to "rich" applications (applications built using AJAX, Flash, Silverlight, etc) - they all "break" the back button.

No longer can a user naturally go back to the previous page (or state) unless RIA applications do a lot of work to enable this seemingly simple interaction.

I just wrote an article on codeproject that describes one way of getting this done, using the YUI JavaScript library.

Vector Spaces

2009-04-23T08:00:00.006+10:00

I was at a semantic conference last year and for me one of the standout presentations was Peter Turney talking about the application of vectors to semantics and how they represent a kind of analog signal versus the digital crispness of AI logic and how you can use both kinds of logic and kind of move between the two to overcome the limitations associated with either one.

I've been using vector space techniques a lot since then and it turns out that they are indeed a useful tool to keep around.

Vectors provide a means to compare multivariate data in a relative way. Sorting on one variable at a time is easy with computers, but to sort by many at once requires some thought. Linear algebra provides some creative techniques to manipulate and gain other insights into your multivariate data, such as singular value decomposition, which leads to latent semantic analysis, wherein a reduction of dimensions can create insight into the data itself.

The other great thing about vector spaces is how imaginative you can be with how you use them. They essentially lend themselves to any kind of comparison or computation. All you need to do is stick your data into the matrix, normalise it intelligently and you can derive patterns and relationships from that data that you wouldn't have seen otherwise.

I started playing around with vector spaces with term/document or document/term matrices - generated by counting the words in a list of documents and putting the counts of each word per document in a row or column of your matrix. Then, so that long documents don't rank higher than shorter documents (with smaller word counts) the values are normalised with a term frequency/inverse document frequency (tf-idf). At which point we can start comparing documents based on the cosine similarity of the document vectors.

Search Result Clustering with Non Negative Matrix Factorisation

2009-03-24T12:03:00.002+11:00

Non negative matrix factorisation. Wow, what a mouthful! Simply put, it's a technique to find two matrices (a features and weights matrix) that when multiplied together will reasonably approximate the original matrix.

Conveniently, the features matrix can be used to group or cluster rows in the original matrix that relate to terms or documents.

An article on codeproject explains it all in graphic detail...

Whole Earth Catalog

2009-03-01T09:48:00.005+11:00

I found a copy of this in my parents house yesterday dating back to 1972 and was it a blast from the past! Definitely a pre-cursor to the semi rambling nature of the internet today, but shaped by a definite editorial bias that is never really made explicit but of certain primary interest to people living in communes, nomads and hippies.

In case you haven't seen one first hand, it is a BIG catalog that reviews products and tells you how to get mail order catalogs from "recommended suppliers" to enable a self-sufficient lifestyle, outside the flow of big bad business entities. I guess that was a very self-empowering concept before the internet existed - that you could live your life without ever dealing with any perceived awful tackiness of malls, of bland advertising, of empty promises.

The "tools" that it recommends are seemingly random, and joined with strange short stories, (or is it a novel?) that tie the pages together. It's one of those things that you either say "genius!" or "what rubbish!" depending on your personal proclivities.

I wonder if the internet has really replaced the notion of a "Whole Earth Catalog" or if there is still a need for an editorialised lense, a sense of common purpose and ideology amidst the unfathomable scale of the internet today?

2009-02-14T16:49:00.001+11:00

Ice Blue Digital has released Wordnet 3.0 Search, a free online dictionary.

There is an obvious paradox to the concept of a dictionary - single words can have multiple meanings and dictionaries use words to define the definition of other words. For example, the humble word "word" has at least ten definitions:

1. a unit of language that native speakers can identify; "words are the blocks from which sentences are made"; "he hardly said ten words all morning"
2. a brief statement; "he didn't say a word about it"
3. new information about specific and timely events; "they awaited news of the outcome"
4. the divine word of God; the second person in the Trinity (incarnate in Jesus)
5. a promise; "he gave his word"
6. a secret word or phrase known only to a restricted group; "he forgot the password"
7. an exchange of views on some topic; "we had a good discussion"; "we had a word or two about it"
8. the sacred writings of the Christian religions; "he went to carry the Word to the heathen"
9. a verbal command for action; "when I give the word, charge!"
10. a word is a string of bits stored in computer memory; "large computers use words up to 64 bits long"

It is only through context that we can understand these definitions, a contextual awareness that has taken years and years for all of us to develop and that depends on everyone reaching more or less the same set of contextual word meanings.

From the perspective of a computer or a non-native speaker, this web of associations can seem arbitary and not immediately apparent. But today arrives a solution!

Wordnet 3.0, combined with the Princeton Wordnet Gloss Corpus has created an exact mapping between each word in the dictionary's definition with the exact semantic definition of the word used in that context.

Try it out and decide for yourself if this free, ad-free dictionary is better than the other dictionaries out there.

Wordnet 3.0 Web Interface

2009-02-09T08:54:00.001+11:00

For those that don't know about it (which usually means people not active in the NLP/Semantic space) Wordnet is a free dictionary created by Princeton that is based on a theory of how the brain organises words. There are numerous semantic links between words, and nouns and verbs are arranged in an ontology that ranges from abstract ideas at the top through to specific ideas at the bottom.

While not really intended as an NLP resource, it has become useful in semantic applications exactly due this hierachy of concepts (if it is known that "a beach house can catch fire" an AI program with wordnet can effortlessly produce the generalisation that "houses can catch fire").

Recently, the same group at Princeton have created a "semantically annotated gloss corpus" (SACG) in which the definitions of each word are cross-linked to the exact semantic definition of each word. This has long been considered a useful exercise as semantically tagged data is in a woefully short supply, and moreover, such a cross-linked semantic dictionary should in theory aid in semantic disambiguation tasks.

Both Wordnet 3.0 and the SACG are now available from a web based interface that combines wordnet lookups and definitions with the full range of semantic links and the SACG, whereby the parts of speech and semantic defintion of each word in the definitions (or gloss) are expressed.

Semantic Symposium at NYU

2008-11-17T00:38:00.001+11:00

Since I was in Manhattan at the time, I attended this really cool event, which featured lots of really smart people basically knocking their heads against the (intractable) problem of AI reasoning based on doing entailment from semantic knowledge, obtained from texts.

It was fascinating to see the attacks that everyone was using to try to slay the dragon. Just about every approach you could imagine was presented. Everyone with something concrete was susceptible to questions - how do you deal with temporality, what about metaphor, etc - to which they would say, "I haven't thought about that yet", or even better, "well we haven't considered these issues yet, but we are about to field test out system with a large company and so we will soon know if it works".

The emperor truly has no clothes.

Still, for someone with lesser goals (a tool to support human reasoning) the mechanics that were presented were fascinating.

Currently Travelling

2008-10-31T05:54:00.000+11:00

My blog posts have transmogrified into travel entries as I am currently circumnavigating the globe. Here for more details...

A Vision of Life

2008-09-03T09:54:00.001+10:00

There's an article on CodeProject that shows a genetic algorithm tuning a neural network to simulate "ants" finding food. The ants start out kind of dumb, just milling around aimlessly (or more accurately, randomly). But some ants randomly have better food collection strategies (small direction changes, fast velocity) that means they collect more food than the other ants. At the end of each "generation" the low scoring ants are unceremoniously removed from the gene pool and the high scoring ants spread their successful food gathering strategies to their ancestors.

It's a grim view of life really. Are we all nothing more than the bearers of a successful food gathering strategy that has been passed down through the generations? Apart from the random droppings of food in the simulation and the ants' disturbing mechanical persistence at finding more and more of it, there is only darkness.

Semantics and Search

2008-09-02T17:43:00.001+10:00

Right now I am working on different projects at Dog Blue Software that revolve around natural language processing and applying it to Internet search. Natural language processing looks at how all the words on a page are used in sentences and giving the computer some idea about what all those words mean (the semantics).

Working with information from the Internet is challenging on a number of levels. There is the amount of data. There are the limitations of the interface. There are the multiple languages and idioms.

Some people talk about "The Semantic Web" as some kind of saviour to the problem of sifting through large amounts of unstructured data. Clay Shirky neatly critiques the limitations of such a concept, even if it were to coalesce into being.

Rather than a simplistic solution - "we'll just ask everyone to become an Internet librarian" - or Google's needle in the haystack solution - "we'll give you ten needles that everybody already knows about, you can forget about all the other ones" - the solution to navigating large information spaces will probably require a level of sophistication bordering on artificial intelligence.

But until then, letting the computer find the meaning of text rather than its representation is an idea that seems to have promise.