algorithm

Thanks to nice illustrative pictures of LSTMs and RNNs by colah’s blog. Recurrent neural networks (RNNs) use the same set of parameters to deal with inputs that are sequential. Inputs are usually broke into pars of same lengths, and fed into RNNs sequentially. In this way, the model learned and preserve the information from sequences of arbitrary lengths. This trait becomes very useful in natural language use cases where a model that is capable of dealing with sentence of any length is needed....

Tree-based methods contains a lot of tricks that are easily tested in data/machine learning related interviews, but very often mixed up. Go through these tricks while knowing the reasons behind could be very helpful in understanding + memorization. Overview of Tree-based Methods Overall speaking, simple decision/regression trees are for better interpretation (as they can be visualized), with some loss of performance (when compared to regression with regularization and non-linear regression methods, e....

In this article, two more methods will be discussed that takes not only linear correlation of a single predictor variable with the dependent variable, but also considers the intertwined effects. These two methods are Commonality Analysis (CA) and Dominance Analysis (DA). They share some similarities yet differs in focuses of analyzing relative importance. Commonality Analysis Idea Commonality Analysis is a statistical technique within multiple linear regression that decomposes a model’s R2 statistic (i....

This article will discuss some problems in implementing multivariate linear regression. These problems are: What to do with the collinearity phenomenon in linear regression? How to extract feature importance of a linear regression, is it just the coefficient of the model? Collinearity What is collinearity Collinearity is a condition in which some of the independent variables are highly correlated. Collinearity will occur more often when the number of predictors is very large, as we can imagine that there is a higher chance that some predictor are correlated with others....

What is Survival Analysis and When to Use It? Survival analysis can be generalized as time-to-event analysis, which is analyzing the time to certain event (e.g. disease, death, or some positive events, etc.). Survival analysis gives us the estimation of, for example: the time it takes before some events occur; the probability of having experienced certain event at certain time point; or which factors have what effect on the time to certain event....

EM And GMM Expectation-maximization(EM) algorithm, provides a way to do MLE or MAE when there is incomplete data or unobserved latent variables. NOTE: EM is general way of getting MLE or MAE estimations, not necessarily for clustering. Gaussian mixture model(GMM) is a statistical model that can serve as a clustering algorithm. It assumes the data points to be from several gaussian distributions and uses EM algorithm to obtain the MLE estimations of those gaussians....

Motivation of Kernel Method In classifications, it is often the case that we want to obtain a non-linear decision boundary. For example, for this problem (figure 2), we want a desicion boundary that is somewhat like a circle, however, our model only yields linear boundaries. In order to let our model to have more flexibility without changing the basic algorithms, we can apply a transformation onto the feature space $X$. like the figure on the right....