Article: "Statistics vs. Data Science vs. BI"

Link: http://smartdatacollective.com/davidmsmith/124376/statistics-vs-data-science-vs-bi

Big Data is About Speed Instead of Just Size

Article: http://www.guardian.co.uk/news/datablog/2013/may/20/big-data-critics-overlooked-speed-not-size

The main point of the article provides an insightful reminder for us not to be fixation with just the volume aspect of big data. There are many cases characterized by a high-velocity stream of data that needs to be processed and acted on immediately.

Google Launches "Mobile Backend as a Service"

Google launches "Mobile Backend as a Service" starter kit. It's called "Mobile Backend Starter" and runs on top of Google App Engine.

It's mainly a client-to-client networking and communications stack. It supports storage of state data on the server side and also Google Authentication. However, no customized server logic or extensions are supported.

References:

• http://gigaom.com/2013/06/03/google-takes-on-parse-with-new-service-for-mobile-app-backends/
• http://googlecloudplatform.blogspot.sg/2013/06/get-your-mobile-application-in-the-cloud-with-mobile-backend-starter.html

Application Management for AWS

Currently, the following application management frameworks are available for AWS (excludes third party solutions):

1. AWS Elastic Beanstalk
2. AWS OpsWorks
3. AWS CloudFormation

Details can be found here: https://aws.amazon.com/application-management/

Something they didn't mention in the above web-page is that OpsWorks is a technology acquired in an acquisition, and does not integrate into all of AWS's services yet, the most important being ELB (OpsWorks uses HAProxy). Leaving CloudFormation as the framework of choice if this is an issue.

Forum post about this: https://forums.aws.amazon.com/message.jspa?messageID=426226

On the other hand, the downside of CloudFormation is the lack of automated failover.

BASH IO Redirection

In bash, by default, only stdout gets redirected to a file when you use the ">" operator.

To redirect stderr to a file, use the "2>" operator.

To redirect both, use "&>"

Reference:http://www.cyberciti.biz/faq/redirecting-stderr-to-stdout/

Git Undo Staging a File for Commit

If a file has already been staged for commit i.e. via the "git add" command, use the following command to unstage the file:

git reset HEAD <filename>

Iterating Through All Loggers & Appenders in LogBack

Most people probably won't need it, but when you do, this will come in handy.

http://stackoverflow.com/questions/7064402/is-it-possible-to-find-logback-log-files-programmatically

The "spi" Package in Java Projects

It's quite common to find a package called "spi" in Java projects.

Here's an explanation of what it means and what it's for:

• http://stackoverflow.com/questions/2954372/difference-between-spi-and-api

C# Iterate Dictionary

Using "foreach" and "KeyValuePair" as the element. Refer to this: http://stackoverflow.com/questions/141088/what-is-the-best-way-to-iterate-over-a-dictionary-in-c

C# Start New Process

This is how it's done:http://stackoverflow.com/questions/258416/shellexecute-equivalent-in-net

If you need to start multiple processes asynchronously, you'd need to first start multiple threads and then within each thread, each process is started.

Reference for this: http://www.codeproject.com/Articles/25983/How-to-Execute-a-Command-in-C

How to add arguments: http://msdn.microsoft.com/en-sg/library/system.diagnostics.processstartinfo.arguments.aspx

C# Getting Time Since Epoch

Many things are simpler in C# than e.g. in Java, but getting the Epoch time is an exception, and this is how it's done.

TimeSpan timeSpan = DateTime.UtcNow - new DateTime(1970, 1, 1); double currentTime = timeSpan.TotalSeconds;

Ubuntu: Configuring Tomcat

Instead of hacking the startup scripts, some of the common configuration variables can be set in Ubuntu in the file "/etc/default/tomcat7"

Git Log Show File List

To list the files that were involved in each commit:

git log --name-status

References:

• http://stackoverflow.com/questions/1230084/how-to-have-git-log-show-filenames-like-svn-log-v

Git Checkout File From Branch

To checkout a file from a specific branch:

git checkout <branchname> <filepath>

References:

• http://stackoverflow.com/questions/8423136/netbeans-git-copy-individual-files-between-branches
• http://stackoverflow.com/questions/307579/how-do-i-copy-a-version-of-a-single-file-from-one-git-branch-to-another

iOS How to Close an App

With multi-tasking, apps no longer really close when you press the home button. It stays in the background and the OS automatically closes them when the device is low on memory and when it thinks the app is unlikely to be used in the near future. To force it to close for e.g. development testing purpose, follow the instructions on this page: http://support.apple.com/kb/ht5137

AWS DynamoDB Java API Gotcha

If you don't set the end-point, it will default to the US East Virginia region, resulting in an error saying the table can't be found.

The error message you'll get varies. If you have IAM setup to limit access for the user account to this specific DynamoDB table, then you'll get an "AccessDeniedException". Otherwise, you'll get a "ResourceNotFoundException".

If your DynamoDB is configured in the other regions, you'll have to set the End Point after creating the client e.g:

client = new AmazonDynamoDBClient(awsCredentials); client.setEndpoint("https://dynamodb.ap-southeast-1.amazonaws.com");

AWS DynamoDB ARN for IAM

To give access via IAM to a specific DynamoDB resource, define the ARN as follows:

arn:aws:dynamodb:<region>:<accountID>:table/<tablename>

AWS IAM Account ID

The account ID is needed to construct the ARN for IAM entries. AWS isn't being very clear about how to retrieve the account ID. It's actually the Account Number (with the hypens removed) that you see under your name in the "Manage Your Account" page (accessed via the "My Account" link in a drop down menu under your name on the top-right of the console).

Mac Show Hidden Files in Finder

I usually don't need this feature, as I can use the Terminal to work with hidden files, but when I needed to restore some hidden files from Time Machine, I needed to get Finder to show hidden files.

Here's a couple of links showing how it's done:

• http://howto.cnet.com/8301-11310_39-20062357-285/how-to-access-hidden-files-to-restore-in-time-machine/
• http://meinit.nl/restore-hidden-files-apple-mac-os-x-time-machine

iOS Simulator: View Files

If you ever need to look into the filesystem of an iOS Simulator to e.g. test file download features, here are some information as to how it's done:

• http://stackoverflow.com/questions/6480607/is-there-any-way-to-see-the-file-system-on-the-ios-simulator
• http://idevhub.com/where-is-the-iphone-simulator-filesystem-stored/