Back to main menu

IT & Engineering

Machine learning for everyday tasks

Machine learning is often thought to be too complicated for everyday development tasks. We often associate it with things like big data, data mining, data science, and artificial intelligence. Sometimes it feels something like this:

PUBLISHED ON

PUBLISHED ON

Machine learning is often thought to be too complicated for everyday development tasks. We often associate it with things like big data, data mining, data science, and artificial intelligence. Sometimes it feels something like this:

Table of content

A brain made up of electrical points

Machine Learning is hard

I have always felt like we can benefit from using machine learning for simple tasks that we do regularly.

Real life example

At Mailgun, we work with e-mail and as part of our offering, we parse HTML quotations. This allows a user to grab the latest reply instead of the entire conversation, which is returned as part of our webhook response. You can read more about how we handle inbound message processing in our documentation.

For those of you who don’t know, here’s what parsing HTML from the public Internet looks like:

Dirty Jeep stuck in the mud

Parsing HTML from public internet

It’s messy and sometimes processes get stuck.

Changing the parsing library can help, but it won’t solve the issue completely because every library has its limitations. You have to restrict the parsing to something reasonable.

Cluster analysis to the rescue

But what should the criteria and threshold be? Should we limit by HTML length or tag count? Maybe both? Maybe by something else? The objective obviously is to process as many messages as possible without shooting yourself in the foot, but the path isn’t super obvious.

That’s where cluster analysis and statistical classification become handy. For the research I was using R, but depending on your task and personal preferences you might use something else – scikit-learnWekaMOA, etc.

Collect a dataset

First, we logged HTML length, tags count, message processing time and put them into a csv file:

The vast majority of messages take fractions of a second to process. So, when collecting the dataset, we had to make sure that we have enough “slow” messages.

We ended up with two csv files, collected on different days. One had 13831 lines and was reserved for analysis and model-training (the train dataset). Another had 12149 lines and was reserved for model validation (the validation dataset).

Generally you want to have at least two datasets – one for training and one for validation. Otherwise you might run into overfitting problem, when your model is well adjusted to the train data but fails in the real world.

Determine the number of clusters

To visualize the data and look for patterns k-means clustering was first tried:

As you can see there is a significant performance improvement up to 4 clusters. After that there is no real boost.

Clustering

The next step was to figure out how the data points get distributed between the clusters:

A clusplot graph

And you can somewhat anticipate the problem already: the clusters form by nipping off the datapoints that are far away, while the interval we’re interested in (1-20 sec) is in the very midst. The issue persists with increasing the number of clusters.

Moreover there is a significant overlap between the clusters in the interval:

A list of clusters code

Compare clusters number 1 and 4. The issue persists with increasing the number of clusters.

At this point, we decided to try a different approach and look at the percentiles for message processing time:

A graph of percentiles

As you can see, after the 78th percentile the processing time quickly bubbles up. Here was our first threshold – 78th percentile that corresponded to 6.5 seconds.

All datapoints that took less than 6.5 sec were marked as “fast” and others as “slow”.

Classification

For classification, we tried SVM (Support Vector Machines), Random Forests and CART(Classification And Regression Tree).

CART showed slightly better results for this task but its main advantage is that it gives you a decision tree that is easy to understand, explain and implement vs SVM or Random Forests that work like a black box and require using heavy ML libraries in production.

Here’s how you classify using CART:

Here’s the decision tree:

Classification tree for message processing time

And that’s how the implementation looks like:

Isn’t it beautiful? All the research complexity in a single line!

Validation

For evaluation we took the validation dataset and tried to predict the classification using our model:

Here’s the confusion matrix and some common classification metrics:

Evaluation code

Lessons Learned

Machine learning is not just for data scientists. Even simple decisions powered by ML can benefit you. Know your data. Do not rely blindly on scientific algorithms and models.

Useful links

Happy machine learning and data mining!

Related readings

Popular posts

Email inbox.

Build Laravel 10 email authentication with Mailgun and Digital Ocean

When it was first released, Laravel version 5.7 added a new capability to verify user’s emails. If you’ve ever run php artisan make:auth within a Laravel app you’ll know the...

Read more

Mailgun statistics.

Sending email using the Mailgun PHP API

It’s been a while since the Mailgun PHP SDK came around, and we’ve seen lots of changes: new functionalities, new integrations built on top, new API endpoints…yet the core of PHP...

Read more

Statistics on deliverability.

Here’s everything you need to know about DNS blocklists

The word “blocklist” can almost seem like something out of a movie – a little dramatic, silly, and a little unreal. Unfortunately, in the real world, blocklists are definitely something you...

Read more

See what you can accomplish with the world's best email delivery platform. It's easy to get started.Let's get sending
CTA icon