Resume SCP Transfer

No way to do in SCP. Use rsync:

rsync --partial --progress --rsh=ssh ec2-user@xxx.xxx.xxx.xxx:abc.tgz abc.tgz

Reference: https://yyab.wordpress.com/2006/12/18/resume-a-large-scp-transfer/

Tomcat Connections Getting Disconnected After Period of Inactivity

In some network environments, the firewall or router might terminate some connections after a period of inactivity.

However, this is not enabled by default:

# netstat -anpo | grep :80 tcp6       0      0 192.168.1.7:80      123.123.123.123:38123    ESTABLISHED 21345/java           off (0.00/0/0)

To prevent this, KeepAlive must be enabled in Tomcat. This is done by editing the Connector configuration in the server.xml file and adding the "socket.soKeepAlive" attribute.

<Connector port="8080" protocol="HTTP/1.1"               connectionTimeout="20000"               redirectPort="8443"                socket.soKeepAlive="true"         />

However, this alone may not be enough, because by default in some Linux distros, the default KeepAlive time is 2 hours, which means the first KeepAlive packet does not get sent until 2 hours later. For networks that disconnect idle connections within e.g. 15 minutes, this is too late. To reduce this to e.g. 5 minutes:

echo 300 > /proc/sys/net/ipv4/tcp_keepalive_time

Note: this change will not survive reboot. To make this change permanent the /etc/sysctl.conf needs to be edited.

After restarting Tomcat, the connection will look like this:

# netstat -anpo | grep :80 tcp6    0      0 192.168.1.7:80    123.123.123.123:38123   ESTABLISHED 21345/java      keepalive (82.52/0/0)

References:

• http://www.tldp.org/HOWTO/html_single/TCP-Keepalive-HOWTO/
• https://tomcat.apache.org/tomcat-8.0-doc/config/http.html

Ansible 2.2 with_items Deprecation

Prior to 2.2, this worked

- name: do something # do something here with_items: ec2.instances

However, this has been deprecated. Changelog item: "with_ 'bare variable' handling, now loop items must always be templated {{ }} or they will be considered as plain strings"

It's now required to do this:

- name: do something # do something here with_items: "{{ ec2.instances }}"

References:

• https://github.com/ansible/ansible/blob/devel/CHANGELOG.md#22-the-battle-of-evermore---active-development
• https://github.com/ansible/ansible/blob/devel/CHANGELOG.md#removed-deprecated

MongoDB: Setup Replica Set

Simple guide: https://blog.ajduke.in/2013/05/31/setup-mongodb-replica-set-in-4-steps/

Additional steps with keyfile auth:

• https://docs.mongodb.com/v2.6/tutorial/deploy-replica-set-with-auth/
• https://docs.mongodb.com/v3.2/tutorial/enforce-keyfile-access-control-in-existing-replica-set/

MongoDB "root" Role

To perform functions such as replication administration, the "dbAdminAnyDatabase" role is not enough. The "root" role is required".

db.grantRolesToUser("admin",[{role:"root","db":"admin"}])

Reference: http://stackoverflow.com/questions/23943651/mongodb-admin-user-not-authorized

Ansible Move File/Folder in Idempotent Manner

Solution is to use conditionals:

- name: stat the dest path          stat: path=/path/dest          register: statdest        - name: move file          command: mv /path/src /path/dest          when: statdest.stat.exists == false

Reference: http://stackoverflow.com/a/24165405/548272

Bash: Easy Way to Tokenize Strings Separated by Space

Using the "set" command, it is possible to replace the position parameters with the one assigned. This can then be used to refer to individual elements of a string separated by space.

$ str="one two three" $ set -- $str $ echo $1 one $ echo $2 two $ echo $3 three

Reference: http://stackoverflow.com/questions/5382712/bash-how-to-tokenize-a-string-variable

Ansible Relative Path with playbook_dir

The playbook_dir resolves to the directory where the playbook file resides. You can use relative path from there to reference other files.

Ansible Variables

Good resource about variables which compiles all the different ways variables are used in Ansible: https://liquidat.wordpress.com/2016/01/26/howto-introduction-to-ansible-variables/

Ansible Variables from File

In the task e.g.

- hosts: appservers  vars_files:        - vars.yml

vars.yml e.g.:

var1Name: var1Value var2Name: var2Value var3Name: var3Value

References:

• https://www.safaribooksonline.com/library/view/ansible-up-and/9781491915318/ch04.html
• http://docs.ansible.com/ansible/playbooks_variables.html
• http://stackoverflow.com/questions/27746077/how-to-include-vars-file-in-a-vars-file-with-ansible

AWS IAM Policy Statement to Register and Deregister Instances with/from ELB

E.g.

{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "Stmtxxxxxxxxx",            "Effect": "Allow",            "Action": [                "elasticloadbalancing:RegisterInstancesWithLoadBalancer",                "elasticloadbalancing:DeregisterInstancesFromLoadBalancer"            ],            "Resource": [                "arn:aws:elasticloadbalancing:aws-region:aws-account-id:loadbalancer/*"            ]        }    ] }

AWS IAM Permissions to Create Tags

Had a hard time getting this to work, and it turns out that only permissions to the "ec2:CreateTags" action is needed, however, the moment the resource is constrained (even to all resources within a specific region), I was unable to tag an EC2 instance. I was only able to tag an EC2 instance when the "Resource" was set to *

{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "Stmtxxxxxxxx",            "Effect": "Allow",            "Action": [                "ec2:CreateTags"            ],            "Resource": [                "*"            ]        }    ] }

Ansible Best Practices: Directory Layout

Here: http://docs.ansible.com/ansible/playbooks_best_practices.html#directory-layout

AWS CloudWatch Logs: Create Log Stream via Command Line

aws logs create-log-stream --log-group-name groupName --log-stream-name streamName

Ansible: Running a Command on the Local Machine

To run an command on the shell of Ansible machine (i.e. instead of the target machine), use the local_action module:

local_action: command xxxxxxxxx

Ansible: EC2 Instance ID Variable

{{ ansible_ec2_instance_id }}

MongoDB: Query for Documents Based On Array Size

db.collectionName.count({"arrayName":{$exists:true},$where:'this.arrayName.length>3'})

Reference: http://stackoverflow.com/questions/7811163/query-for-documents-where-array-size-is-greater-than-1

Ant Java Compilation Classpath

If there are many jar files in the classpath, instead of specifying all of them, include all jar files in a folder this way: http://stackoverflow.com/questions/6103516/ant-adding-multiple-jars-in-classpath-dynamically

Amazon Linux Install Ansible

The YUM method won't work, due to differing python versions.

Use pip instead.

sudo pip install ansible

Passing Environment Variables into Java from Command Line

java -DvarName1=varValue1 -DvarName2=varValue2