Finance data feeds

http://quant.stackexchange.com/questions/141/what-data-sources-are-available-online

Zen Stats

From a coworker:

Model knows nothing, just the signal that comes.

Neural Networks

I wondered how they were doing the unsupervised learning that everyone has been writing about:

http://deeplearning.stanford.edu/wiki/index.php/Main_Page

I think the gist is: use the input as the desired output, and let the network figure out the concept. How very recursive and GEB. :)

And of course, NN as a service: NNAAS? NASTY? Something. Could be a good idea if their pricing is right.

http://www.ersatz1.com

Beta-Surface

This is another visualization of the predictive model I'm working with.

From front to back on the X axis are ranks of subjects (ranked by probability of purchase).

On the Y axis (from left to right are individual variables in the model).

Z axis is the mean xbeta, (basically variable * weight).

I've colored them by their standard deviation, so grey-orange-red is pretty low variation.

The 3D isn't that great (I'm still figuring out gnuplot), but it's still super interesting to see what's going on inside the machine. 

i-Integration and Telemedicine

I had a dream last night that I had a telemedicine app on my iPhone.

After coming across a woman in labor I opened my telemedicine app that connected me with a live doctor to help deliver the baby. I was given the prompt: "This service will cost $279. ok?, Cancel?"  I do believe I debated if $279 was too much to spend on a stranger. Obviously dreamland mechanics were at work, but an interesting idea.

Then this morning, I realized that Siri, my Philips Hue smart lightbulbs, and the Nest learning thermostat should all work together.

"Siri, turn on the lights when I get up in the morning". 

Now, notice what I'm asking here: I'm not asking for the lights to turn on automatically at 7am, I'm asking for Siri to wait to detect my presence at around 7am via the Nest (or other) motion sensor, and THEN turn on the lights. Like a real smart-house should. 

From time to time I hear someone prognosticate that this is it, we've come to the end of this fantastic age of advancement:

http://www.nytimes.com/2012/12/28/opinion/krugman-is-growth-over.html

These people should be retired immediately from any sort of creative position. If this is the extent of your imagination, then you need to get out of the way.

Probability of Purchase

Just thought I'd send this out for all to see.

This is a visualization of a model of probability of purchase I'm working on.

It's a 4D graph (3D heat map), that shows:

On the X(Front to back) are individual subjects.

On the Y(Left to RIght) are variables used in the model

Z is their Daily Hazard ( chance that they will purchase in the day )

And the color is a heat map denoting the value of the individual terms on the Y axis.

Our model is a Logistic Model, so it's a sum of each term (on the Y).

On the Right is the intercept term, which is constant, very negative (purple).

In the hockey stick section you can see terms that were negative start to swing positive, and way over on the left, there is one term that lights right up on the hockey stick. 

Very fun stuff.

Head 'esPLODE!

Pages like this make we simultaneously happy and sad.
https://en.wikipedia.org/wiki/Categorical_distribution

And the nice write up:
http://izbicki.me/blog/the-categorical-distributions-algebraic-structure

If only I had more time….

Pupilometry

"Princeton University psychologist Daniel Kahneman showed several decades ago that pupil size increases in proportion to the difficulty of a task at hand. Calculate nine times 13 and your pupils will dilate slightly. Try 29 times 13 and they will widen further and remain dilated until you reach the answer or stop trying."

http://www.scientificamerican.com/article.cfm?id=eye-opener-why-do-pupils-dialate

More fuel for my belief that I need to get off my ass and write some serious gaze tracking/pupil tracking software for smartphones.

Death to Spring!

http://blog.jelastic.com/2013/01/03/new-years-resolutions-for-the-java-developer-guest-post/

"I'm tired of the "For Real?" look I get from my developers when they are debugging Spring configuration and they ask my why Spring is better."

"I'm tired of the "For Real?" look I get from my developers when they are debugging JPA/Hibernate issues and I'm explaining to them how JPA/Hibernate simplifies things."

I've been thinking about a Spring Blog post for some time now, and It started back in Feb '12 when I started my new job. There are quite a few Jr. Engineers there, and after noticing they didn't use Maven or Spring, I paused for a second before doing an overhaul. Maven would be nice, and we could certainly use some dependency management, but Spring.. oh Spring…

After being bitten by the chaos, bloat, and sadness that is Spring I had to think long and hard about why I liked Spring in the first place.

Testing.

I can just hear my old co-worker now: "How are you going to test without Dependency Injection?!" 

Besides some nice wrappers around JDBC and a few other APIs, testing is really the only useful thing that Spring did for me. Lets be completely honest, there are only two cases where I want to significantly reconfigure an application: Production use, and Unit Testing (everything else is just a config param change, for which Dependency Injection is WAY overkill).  So stopping to think for a minute I ended up with a scheme where most of my dependencies were created in protected methods, and my unit tests simply created anonymous classes that override the required protected methods. Done.

BAM!

And just like that, no Spring, no wacky half supported ASM / Byte code generating, voodoo that is impossible to debug, breaks when mixed with the wrong framework, bloaty, buggy, complexity. Just override the protected methods and you are testing like a pro. Works great with final variables, you don't have to have a constructor 20 parameters long, no pointless setters, no mystery stack  traces, works with all coverage tools,  just get your job done and move on.