Jack Dermody

Machine learning

Introducing Bright Wire

Symptoms may include: talking about machine learning with anybody that will listen, ignoring bored expressions on people's faces while further continuing to talk about machine learning, creating weird diagrams on napkins at social events - also, creating open source machine learning libraries on GitHub. Lately I've caught the machine learning bug. Before I started learning about machine learning I was interested in natural language processing (actually I still am). Now it's just a matter of training a machine learning algorithm (of which there are many flavours) on what we know are the correct answers. Or, at least it was before machine learning. Anyway, there didn't seem to any other machine learning libraries that did what I wanted. One definition is that machine learning is “the automation of automation”. Machine learning turns traditional programming upside down. So when we want computers to make diagnoses from medical scans or translate text between multiple languages, machine learning is the only good tool that we have. For any non-trivial problem that you need to solve, machine learning might be able to figure out the best way to do it - without you needing to write any software yourself.Not writing software means that you can go to the beach while the computer works out the tricky parts of your process. If machine learning can do the hard stuff, it can also do the easy stuff.

Classification Overview with Bright Wire

The machine learning equivalent of Hello World is probably the Iris data-set, so lets use that. Now that we have our data, let's do some machine learning! The classification accuracy is around 97% from one of the simplest machine learning classifiers. Naive Bayes and Decision Trees aren't linear algebra based machine learning algorithms (they don't use vectors or matrices).

Introduction to Bright Wire

Bright Wire is an open source machine learning library for . It includes the most commonly used machine learning algorithms such as Linear Regression, Logistic Regression, Naive Bayes and Neural Networks (feed forward, convolutional and recurrent). Most included machine learning algorithms can be trained on the GPU for improved learning performance. Once your machine learning models have been trained you can execute them using highly optimised linear algebra libraries such as the Intel MKL library - or even on the GPU - all from within the same library. Bright Wire is an end to end machine learning solution, covering both model training and efficient model execution in production. Bright Wire makes it easy to experiment with machine learning with the performance and type safety of managed languages such as c#. Treating a machine learning algorithm as a composition of nodes makes machine learning almost analogous to snapping together LEGO bricks. This means that any machine learning algorithm that uses linear algebra can run on either the CPU or GPU. Bright Wire supports GPU based machine learning by implementing every linear algebra operation twice - once for the CPU and once for the GPU. The base library includes all machine learning algorithms and CPU based linear algebra: Data tables are the single way to describe data for machine learning in BrightWire. Different machine learning algorithms expect data tables with different column types. Some machine learning algorithms expect input in the form of a list of numbers with an associated classification label (for example the multinomial naive bayes classifier). Follow the tutorials to learn more Bright Wire and machine learning.

Sequence to Sequence with LSTM

For machine learning tasks involved with classifying sequences of data there might not be a one to one mapping between input and output classifications. Since Bright Wire is a graph oriented machine learning framework, the graph for the LSTM network is implemented as:

Extending Bright Wire: Custom Activation Function

So it's clear that there are many possible approaches in machine learning and deep feed forward networks with SELU is indeed a tool worth considering.