Friday, September 01, 2017

Splitting a Space Separated Dataset with awk

I thought I would publish a simple script on how I split a dataset that looks like below into a CSV file which Excel could work on. The issue I was having was that Excel couldn't seem to find the spaces and use them. I would end up with extra data in columns on my Mac. Perhaps a flaw in the Excel program, or a flaw in the data, or both. However, awk digested it just fine. So here is my script to make it a little more useful for Excel. Mind you it is simple, but there are 1.7 million lines of logging, and it took about a second on my machine to convert. Slick by any measure. The problem now is that Excel is choking on the number of records to display in a chart.

Wednesday, August 09, 2017

Docker and Apple Server Service: How to use ports 80 and 443 on Mac OS X

Introduction

I was trying to deploy some Docker containers yesterday which use ports 80 and 443. OK, I will confess that I was trying to deploy Wordpress or Bitnami Wordpress and MySQL to containers to see if I could migrate my personal blog to Wordpress. Eventually, I am hoping to migrate all of my blogs to a new blogging environment.

Problem

Well the containers would not deploy because the ports 80 and 443 were being used. A quick connection to localhost confirmed that the Apple Server.app was using these ports for running its processes. So I logged into the Server.app only to discover no way to turn it off.

Solution

Apple Server.app service is simply that... a service. The launchctl command will allow us to stop and start services. So I tried to stop the service only to discover it will automatically restart on a new PID. The only solution apparently is to unload the service temporarily. The following commands will allow you to unload and load the com.apple.serviceproxy service, and check its status. This will allow you to use Docker containers on those ports while doing your development and testing.

Sunday, August 06, 2017

Apache Tomcat Container Managed Security and HTTP Security Headers

Secure j_security_check Response Headers

Introduction

I was recently tasked with resolving a security issue with Apache Tomcat. The issue was that a series of Filter classes that were implemented in the web.xml were being bypassed when we are using container managed security, e.g. j_security_check. The filters were adding security headers:
  • X-Content-Type-Options: nosniff
  • X-Frame-Options: SAMEORIGIN
  • X-XSS-Protection: 1; mode=block
When the response is returned, it would include these values. The filters were written prior to the org.apache.catalina.filters.HttpHeaderSecurityFilter. They duplicate the functionality, but the Apache version has a few more features. The Apache version is generally implemented in the web.xml file for the whole server, e.g., <CATALINA_HOME>/conf/web.xml. The code is really well written, and I would recommend using it except if you are using container managed security.

Issue

Container managed security,j_security_check, is implemented using a Valve. In particular, this intercepts the call and returns the form based login page which contains something like this: The resulting response does not contain the headers noted above using either our custom filters, nor from the Apache HttpHeaderSecurityFilter. Our automated security testing software, OWASP Zap caught it.

Solution

The easiest solution I came up with was to implement a couple of valves that add the selected response headers. I then added the Valve implementations to the <CATALINA_HOME>/conf/context.xml. Adding them to the default context.xml file allows them to be used on all applications deployed to the server. If you don't want applied to every application, you can add it to the context.xml file in the individual project.

The project can be found on Github here: tomcat-security-valves.
<dependency>
  <groupId>com.bluelotussoftware</groupId>
  <artifactId>tomcat-security-valves</artifactId>
  <version>1.0.0</version>
</dependency>

An example application using the default Apache Tomcat realm is available here: tomcat-container-managed-security

XContentTypeOptionsValve.java



XFrameOptionsValve.java



XSSProtectionValve.java


Popular Posts