In the first installment of Snake_Bytes, Ted brought to our readers' attention the dot product, which prompted a reader question about vector multiplication, and... fruit. To illustrate an application of the dot product in statistical modeling, it is useful to consider a vector as a collection of *n *coordinates (also sometimes referred to as attributes, parameters, or features) that define the location of a **point** in \( \mathbf{R}^n \). With a little rearrangement of equation 2 from Ted's post, we see that the dot product of two L2-normed vectors is equal to the cosine of the angle between those points:

\[cos(\theta) = \|A\|\cdot\|B\|\]

The normalization bounds the inner product in [-1, 1], which makes this metric convenient for interpreting object (point) similarities.

To take this a step further, rather than thinking about * quantities* of fruits as posed in Ms. Dalton's question, imagine instead we need to create a classifier that allows a computer to easily distinguish between two very different kinds of fruits: apples and bananas. Since the typical colors of bananas and apples differ so strongly, we can use a simple device that measures only 2 properties – the "redness" and the "yellowness" – of a large sample of known apples and bananas. Let's use Python to generate a table of what such measurements may look like:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
#some basic imports import numpy as np import pandas as pd #these are our fruits fruit_names = {0:'a banana', 1:'an apple'} #fake some univariate normal fruit sample color data n_pts = 1000 fruit_samples = pd.DataFrame(np.random.randn(n_pts,4), columns=['ban_red','ban_yel','app_red','app_yel']) #make the banana and apple color samples more yellow and red, respectively fruit_samples[['ban_yel','app_red']] += 5 #our fictional measurement device doesn't support negative values fruit_samples[['ban_red','app_yel']] = abs(fruit_samples[['ban_red','app_yel']]) |

Note that in this highly simplistic 2-dimensional model we are assuming that (ripe) bananas are mostly yellow, whereas most apples are red. For the sake of convenience, we've structured the ** fruit_samples** matrix with the 1000 observations going down the rows, and the [red, yellow] measurements for bananas in the first two columns, followed by the measurements on apples in columns 3 and 4. With these measurements, we can define the 'exemplar' banana and apple as the average of the red and yellow values across the set of both fruits in our sample.

1 2 3 4 5 6 |
sample_means = fruit_samples.mean() THE_banana = np.array(sample_means[['ban_red','ban_yel']]) THE_apple = np.array(sample_means[['app_red','app_yel']]) #L2 norm the model fruit vectors fruit_model = np.vstack((THE_banana/np.linalg.norm(THE_banana), THE_apple/np.linalg.norm(THE_apple))) |

Using the average of our measurements allows us to easily use dot products when we want to classify new [red, yellow] measurements as either bananas or apples: intuitively, if a test vector is more yellow than it is red, it should be 'closer' to the exemplar banana, which in turn will be reflected in the cosine similarity score. The following plot shows all 1000 samples of bananas and apples plotted according to the [red, yellow] vector measured on each, along with the exemplar banana and apple shown as a large diamond and circle, respectively.

Now we can create a function that takes as input a test [red, yellow] color vector, the dictionary that identifies the order and names of our sample fruits, and our model fruit matrix. We use the numpy library's dot function to compute the cosine similarity, * fruit_similarity*, between the input

*vector and the*

**test_fruit***matrix:*

**fruit_model**
1 2 3 4 5 6 |
def estimate_fruit(test_fruit, fruit_names, fruit_model): import numpy as np test_fruit_norm = np.asarray(test_fruit)/np.linalg.norm(test_fruit) fruit_similarity = test_fruit_norm.dot(fruit_model.T) fruit_est = fruit_names[np.argmax(fruit_similarity)] return {'fruit_estimate' : fruit_est, 'fruit_sim' : fruit_similarity} |

Let's have a look at the resulting * fruit_similarity* array given a test_fruit input of an apple with [red, yellow] vector [0.9, 0.1]:

1 2 3 4 5 |
test_fruit = [.9, .1] my_estimate = estimate_fruit(test_fruit, fruit_names, fruit_model) my_estimate {'fruit_estimate': 'an apple', 'fruit_sim': array([ 0.26274262, 0.99885659])} |

The numbers in this 2D array correspond to the cosine similarity scores between our test vector and the banana in the 0th element of the array (0.263) and the apple in the first element (0.999). We then simply use the numpy argmax function to return the index of the largest value from this array, pass that index to our ** fruit_names **dictionary, and it returns the expected

*: 'an apple'.*

**fruit_estimate**

With only two dimensions, it is a bit difficult to realize the power of this method... one could easily just pick the index of the color value that is largest and call that the winner. But what if we had yellow and green apples in addition to red, or unripe bananas? Better yet, what if we wanted to create a program to distinguish between **all different kinds** of fruits? We would clearly need a more sophisticated model, based on a higher-dimensional set of observed features such as color, shape, texture, size, etc. Now the power of a dot product becomes more evident: we would be able to compare test vectors against all the features in this higher dimensional model simultaneously, still producing accurate predictions based on this larger feature space. However, if the space gets too large, and/or the sample vectors are sparse, some interesting problems arise. We will explore the consequences of these effects in future posts.

- On The Necessity of Blockchain - October 7, 2016
- Snake_Bytes #4: A Simple Classifier - September 27, 2016