Search This Blog

Friday, October 23, 2015

ZSH and Oh-My-Zsh Shell Plugins

I remember a long, long time ago, in a galaxy far, far away, I played around with setting up custom DOS prompts.  Memories of Ansi.sys and custom ANSI art come streaming into my brain...



Forget all that.  On CentOS, these two commands will install the Z Shell and Oh-My-Zsh

yum install zsh
sh -c "$(wget https://raw.github.com/robbyrussell/oh-my-zsh/master/tools/install.sh -O -)"

Tab allows you to visualize potential paths, running processes, ls without hitting enter, and other awesomeness.

There are one or two Themes and Plugins available.

Some laundry lists, tricks and cheat sheets.

If you're running Windows, 720MB of Babun will get you Zsh among other things...

Sunday, October 4, 2015

Hue on HDInsight and HDInsight on Linux

Microsoft might have just made Data Lakes a commodity offering.

Convergence with the Linux realm is happening again at Microsoft with the introduction of Hue on HDInsight (a graphical interface for Hadoop/HDP) and HDInsight on Linux. Hue has been around for quite awhile in the Apache realm and in most Hadoop distros, glad do see HDInsight is finally getting a user-friendly GUI.

Another announcement introduces U-SQL (see Michael Rys (@MikeDoesBigDataIntroducing  U-SQL).  A SQL-like, Hive/Pig/Grep/Awk combo language to ELT+QE (Extract/Load/Transform + Query/Extract) on top of the HDInsight Big Data Lake.

The biggest announcement is the Azure Data Lake itself...




Shared folders with Virtualbox and Centos 7

Got a build error with VirtualBox add-ins and HDP Sandbox 2.2.

Building the main Guest Additions module   [FAILED]

Fixed by checking the log file for errors,  Missing kernel directory issue.

$ export KERN_DIR=/lib/modules/2.6.32-504.1.3.el6.x86_64/

$ cd /usr/src/kernels

$ ln -s /usr/src/kernels/2.6.32-573.7.1.el6.x86_64/ 2.6.32-504.1.3.el6.x86_64

Thursday, September 24, 2015

Multi tab Putty

Just as it sounds, multiple tabbed putty.
http://ttyplus.com/

And for Windows, there's Clover
http://ejie.me/


Sunday, September 20, 2015

Scala kernel for Jupyter notebook and randomness

Some links for Jupyter Notebook setup and randomness...

The Jupyter Scala kernel from Alexandre Archambault
https://github.com/alexarchambault/jupyter-scala

The Jupyter Spark kernel from Brian Schlining
https://github.com/hohonuuli/sparknotebook

Something random from Bryan - convert Genome information to Midi music.
https://github.com/hohonuuli/dna-music/blob/master/README

Something else, video to ascii with akka streams?
https://github.com/hohonuuli/streamerz

Jupyter Server
https://github.com/jupyter/jupyterhub

Setting up public server
http://jupyter-notebook.readthedocs.org/en/latest/public_server.html

Java 9 Kernel for some REPL prototyping
https://github.com/Bachmann1234/java9_kernel
http://blog.takipi.com/5-features-in-java-9-that-will-change-how-you-develop-software-and-2-that-wont/

Adding R to Jupyter
http://ihrke.github.io/jupyter.html

Saturday, September 12, 2015

Jupyter, iPython and AVG Antivirus

Linux tools like cygwin and python don't play well with security and Windows.

Not sure why I bother trying to work with linux apps on a wintel box anyway, but if anyone else is...

Exclude c:\python34 directory in virus scanner
http://useragent.xyz/lost-and-important-file-for-python-34/
to
https://try.jupyter.org/
by running
pip install jupyter
for
https://github.com/zabirauf/icsharp
by running
choco install icsharp
which installs Python 3.4.3... sigh
and throws some 404 error
and installs 2 / 4 packages,
not including icsharp.

Looks like http://python-distribute.org/distribute_setup.py is for sale.  :)

git clone https://github.com/zabirauf/icsharp

If you're not familiar with iPython, now called Jupyter to be language agnostic, it is pretty awesome and distributed peer programming will never be the same.
http://www.nature.com/news/ipython-interactive-demo-7.21492

https://www.authorea.com/
https://cloud.sagemath.com/
https://wakari.io/

Now if I could only get the shift-enter compile execution shortcut working with OneNote and C#,  I would be so happy.
http://tryroslyn.azurewebsites.net/



Monday, August 17, 2015

Engine Noise and the Internet of Things

According to a recent blog post by Stephen Few, Data Visualization Guru,  "The exponential growth in raw data that we’re experiencing is mostly producing noise."

I used to be a car audiophile of sorts.  It was mainly about the highest tweets and lowest subs.  Surprisingly my hearing didn't get permanently damaged, though I did crack a windshield and shake off my rear view mirror a few times.  I still have one of these in my garage...

On a recent road trip, I introduced my kids to Pink Floyd's Dark Side of The Moon.  I realized during the intro to Money that the right half of my speakers weren't putting anything out.  I hadn't noticed prior to this, since most of the music I listen to now is on the radio and is really just noisy filler for my commute. 

Unlike my faulty right door speaker, a pilot might notice more if the right half of their aircraft wasn't putting out any power.  I don't think they would even need any instruments to tell them something is wrong.  A few years ago, there was a frequently quoted IoT statistic put out about Boeing 787's creating a half-terabyte of data per flight.  That's about 12,500x the amount of data a plane from 1977 might generate

After all this data is generated, the results need to be interpreted by the plane's computer, the pilot, and ground crew and actioned on.  Sometimes in real-time, sometimes even predictively.  Perhaps 85% of the data could be considered noise.  That's still about 75GB of data to scan for each flight.  If it's text data it could be shrunk down to under 10GB.  Not quite big data anymore if we get rid of the noise.

I couldn't find a sample of this data, though I did find this report on Noise data for the first 17 months of Boeing 787 operations at Heathrow airport

According to the study, the Dreamliner is 3-8db quieter than similar aircraft.  That's about the equivalent of someone breathing, though I guess at sustained time intervals and multiplied by the number of aircraft in flight it could make a difference.  The study might have been helped (or hindered) by including some visual and audio samples for reference.

Rather than capturing hundreds of statistics and spending months and countless dollars studying flight patterns, a great gig in the sky, the better metric might have been "is it quieter than the Concorde?"  

 
 

 


 
 

How can we determine signals from so much noise?