Comments on: Principal Component Analysis: The Lovecraft Experiment (part 1)

By: mentatjack

mentatjack — Tue, 06 Oct 2009 02:16:00 +0000

The matrix that I start from has one row per book and one column per tag. From that compute the covariance matrix and eventually the principal components. The plot is the first and second principal components mapped against each other. The third-fifth principal components contribute respectively to the RGB values of the point. I divided these into "plots" based on a "pivot tag" as displaying 65,000 books proved a bit unmanageable. That's where the "Master Cloud" came from. I've made numerous other posts on this blog related to the TagShadow project and I've also started collecting my thoughts on the TagShadow forum. It's a pleasure to converse with someone who understands this (probably more than me). I'm by no means an expert on any sort matrix based analysis, be it PCA or MDS, but it's been much fun plugging away at it and getting a chance to stretch my Java development muscles.

By: James

James — Tue, 06 Oct 2009 02:03:15 +0000

The TagShadow mapes look very interesting. Are the positions calculated with PCA? It would also be interesting create a
2D map for these 1500 tags to see clusters among the tags. To create such map you just need to transpose tag table.

By: James

James — Tue, 06 Oct 2009 01:46:12 +0000

Most MDS methods operate on the distance matrix, similar
as PCA operates on covariance matrix. There should be no
problem with high dimensionality, at least not more so than PCA.
The number of data points is rather a limiting factor, at it
decides the dimension of the distance matrix.
Some more recent developements in MDS run under term “manifold learning”, you can find more interesting stuff through google.
Anywhere, I am lookking forward to your part2 and part3
blogs about PCA.

By: mentatjack

mentatjack — Mon, 05 Oct 2009 23:10:03 +0000

Thanks for the heads up! Just a quick glance at the MDS Wikipedia Entry states that this is rather cumbersome for more than 20 variables. My current TagShadow Prototype uses 1500 variables and PCA seems to handle it fairly well. I'll probably explore the results of this particular analysis for awhile before starting from scratch with a new one. I'd love your input on the project.

By: James

James — Mon, 05 Oct 2009 21:34:23 +0000

You might be interested in looking into methods termed as
“multidimensional scaling(MDS)” which seem to address
the same problem as yours. PCA is just one of linear MDS methods.

By: Pricinipal Component Analysis: The Java Iteration « MentatJack

Pricinipal Component Analysis: The Java Iteration « MentatJack — Sat, 12 Sep 2009 19:32:00 +0000

[...] Component Analysis: The Java Iteration Much more satisfying than my earlier attempt to do this by hand, I’m actually building the tools that will eventually power TagShadow. [...]

By: Tagshadow: Jama, Weka and Scatter Plots « MentatJack

Tagshadow: Jama, Weka and Scatter Plots « MentatJack — Fri, 11 Sep 2009 16:53:15 +0000

[...] some progress with Jama, which allows me to represent and manipulate matrices in Java. I ran my lovecraft example data through the paces of SVD, QR, LU, and such, but the real test will be when I try and manipulate the [...]