Posts

Showing posts with the label Model

A Basic Recipe for Machine Learning

Image
Ever since wrapping up the three Deep Learning courses by Andrew Ng I've been meaning to write down some of the gems that he's highlighted throughout the course. One of the nice ones that I felt needed to be written down is his general recipe to approaching a deep learning algorithm/model. I've basically summarized it in a flowchart below (because everybody loves a flowchart right?) Basic deep learning recipe What is bias and variance? The below diagram is the typical explanation that I'm sure most of us are used to.  Graphical illustration of bias and variance How can we know if we have high bias or high variance?  For high bias, we could take a look at the training set performance. A poor performance is an indicator of a poor model fit, and signals that we could try to apply a bigger network to get a better fit of the model. For high variance, we could take a look at the validation set (or dev set - as Andrew calls it) performance. ...

Assign select result to variable in Netezza stored procedure

Image
Now THAT is a lengthy title for a blog post. Am currently working on stored procedure to calculate Dijkstra's shortest path when I ran into this problem (as stated above). Looked through Netezza's Stored Procedure guide but couldn't find anything of use (perhaps I was not looking hard enough. Unfortunately for me even more when most of the SQL-variants out there also couldn't point me in the right direction (even PostgresSQL!) I blame you for not being able to sleep tonight! After examining the error code in Aginity multiple times, I tried to infer that the INTO probably had to be put after the statement, since the error message was complaining something about not being able to do select a variable before doing an INTO. So what if the variable was put after the INTO? Maybe even after the whole statement itself. DECLARE vID varchar; vESTIMATE integer; ... ... select '5','8'--id , estimate, from  ( select row_number...

Person Movement Prediction Using Hidden Markov Models

Humans typically act in a certain habitual pattern, however, they sometimes interrupt their behavior pattern and they sometimes completely change the pattern. Our aim is to relieve people of actions that are done habitually without determining a person’s action. The system should learn habits automatically and reverse assumptions if a habit changes. The predictor information should therefore be based on previous behavior patterns and applied to speculate on the future behavior of a person. You can get the complete article here .

Modelling the Ebola Outbreak using Wolfram

The recent outbreak of the Ebola virus disease (EVD) has shown how quickly diseases can spread in human populations. This threat is, of course, not limited to EVD; there are many pathogens, such as various types of influenza (H5N1, H7N9, etc.) with the potential to cause a pandemic. Therefore, mathematical modeling of the transmission pathways becomes ever more important. Health officials need to make decisions as to how to counter the threat. There are a large number of scientific publications on the subject, such as the recent Science publication by Dirk Brockmann, which is available here. Professor Brockmann also produced videos to illustrate the research, which can be found on YouTube (video1, video2, video3). It would be interesting to reproduce some of the results from that paper and generally explore the subject with Mathematica. Full article here:  Modeling a Pandemic like Ebola with the Wolfram Language

Models

Image
When we talk about big data analytics, the usual topic of discussion would normally revolve around Hadoop, Pig, MapReduce, NoSQL - platforms basically. Granted - those technologies are the enablers of big data, for without which - one can't store big data in their data warehouse. For now though, I'd like to focus on the math. The models to be precise. For without which, one can't derive any usable use cases anyhow with all those data that you have (you might be able to get those low hanging fruits, but over the years - you will have to put on your math hat as well). Below is a link for a course in Coursera which I find to be perfect for beginners like myself who's rather new with statistics and the different models that can be used to solve different questions. The reason I'm mentioning this course in particular is because I like the pace at which it is going and how the presenter is able to articulate complex ideas in simple words. https://class.courser...