Thursday, October 12, 2017

Article About Business Rules Versus Machine Learning

We usually program logic using business rules. However, there are cases where business rules are not clear cut and need continuous iterations to get right. This can be costly and difficult. In these cases, the machine learning approach might help.

Article: https://www.linkedin.com/pulse/data-science-machine-learning-vs-rules-based-karthik-guruswamy/

Tuesday, February 14, 2017

Resume SCP Transfer

No way to do in SCP. Use rsync:
rsync --partial --progress --rsh=ssh ec2-user@xxx.xxx.xxx.xxx:abc.tgz abc.tgz

Reference: https://yyab.wordpress.com/2006/12/18/resume-a-large-scp-transfer/

Thursday, January 12, 2017

Tomcat Connections Getting Disconnected After Period of Inactivity

In some network environments, the firewall or router might terminate some connections after a period of inactivity.

However, this is not enabled by default:
# netstat -anpo | grep :80
tcp6       0      0 192.168.1.7:80      123.123.123.123:38123    ESTABLISHED 21345/java           off (0.00/0/0)

To prevent this, KeepAlive must be enabled in Tomcat. This is done by editing the Connector configuration in the server.xml file and adding the "socket.soKeepAlive" attribute.
   <Connector port="8080" protocol="HTTP/1.1"
              connectionTimeout="20000"
              redirectPort="8443"
               socket.soKeepAlive="true"
        />

However, this alone may not be enough, because by default in some Linux distros, the default KeepAlive time is 2 hours, which means the first KeepAlive packet does not get sent until 2 hours later. For networks that disconnect idle connections within e.g. 15 minutes, this is too late. To reduce this to e.g. 5 minutes:
echo 300 > /proc/sys/net/ipv4/tcp_keepalive_time
Note: this change will not survive reboot. To make this change permanent the /etc/sysctl.conf needs to be edited.

After restarting Tomcat, the connection will look like this:
# netstat -anpo | grep :80
tcp6    0      0 192.168.1.7:80    123.123.123.123:38123   ESTABLISHED 21345/java      keepalive (82.52/0/0)

References: