Elasticsearch analogies

I work a lot with Elasticsearch.

At my job we have migrated from using SQL Server to Elasticsearch for log storage.

Sometimes, when you're learning new grammar for a new technology you're working on it's nice to have metaphors or similes to use when learning the component parts.

One of the best similies I've heard of for what Elasticsearch represents is that it's like an index at the end of a book. Additionally, Elasticsearch by many, even those who work at Elastic, consider it not a DB.

If you start using Elasticsearch a lot more you'll begin to work with types, templates and mappings.

A type could be analogous to a regular SQL table.

Mappings could be similar to a a table column that stores a field a certain way that could be something like a string, int, varchar.

How mappings and settings are created and loaded has changed from Elasticsearch 1.x to 2.x+. Now that we use Elasticsearch 2.x we have to load template files that are used when new indices are created. They combine settings and the mappings for the fields that get parsed out and placed into an index.

Thanks to this website for giving me those Aha! analogies.

http://swops.com/blog/elasticsearch-mappings-and-templates

Rebuild project of 18 year old Mansfield Toilet

I just completed a toilet rebuild last weekend. The toilet began leaking a year or two ago; especially during warm weather.

Although I wasn't working the whole time the total time from start to finish was a half day. I did initial work and investigations starting Friday night, I let the tank dry out overnight, and I finished about 1 PM the following day having started around 10 AM with the installation of the new parts on the tank.

Actual time working was probably around 4 hours.

Work entailed replacing the fill valve, flush valve, new bolts attaching the tank to the bowl, new shutoff valve and supply line too.

New shutoff valve

New Supply Line to Tank

All parts replaced in dry tank

Tank full with water

All parts were ordered from Mansfield and the cost was just under 50 bucks. I went to Home Depot for the new shutoff valve and supply line.

Total cost was about 60 bucks.

Issues encountered after the replacement was slight leaking from one of the bolts, which I had to tighten down more to prevent further leakage.

As mentioned on my Twitter post, I'd say this home project was a 2-3 out of 10.

Perl : subroutine redefined, prototype mismatch

I was refactoring my code and placing some common methods into a base module in my Perl lib directory.

Unfortunately some of my new method names had name collisions with possible subroutines in other packages.

I was able to rename the methods that I was exporting for use in my PL files and the errors went away.

Python Indentation Error

I've only been using Python for a year.

Before that, my focuses were Java and Perl.

An interesting error I came across, since Python has stricter criteria for formatting was: "IndentationError: expected an indented block"

The code in question:

The error might be considered a red herring since the problem with the else: statement was that there was no real code to run after the if statement.

Once I added a logging statement:

The error went away.

Setting up a new hyper-v Windows 2012 server to be pingable and allow RDP connections

At my current company we are needing to test deployments not only to Linux but Windows too.

To quickly have preconfigured systems come up we use virtualization (Hyper-V) along with other technologies like Vagrant and Ansible with other configuration and testing scripts like Python and/or Powershell.

I just setup a new Windows 2012 R2 Machine in Hyper-V using a ISO image.

By default, some features aren't enabled to make my testing easier. For example, to be able to ping the machine or RDP into it for GUI related tasks.

To enable Windows to respond to a ping you have to go into Windows Firewall with Advanced Security > Inbound Rules and enable File and Printer Sharing (Echo Request - ICMPv4-In) by right clicking on the rule name.

For remote desktop connections I had to do two things:

1) Type SystemPropertiesRemote and select "Allow remote connections to this computer."  I also deselected the "Allow connections only..." beneath it

2) I ran this command: netsh advfirewall firewall set rule group="Remote Desktop" new enable=yes

If for some reason I disabled Allow remote connections again I would have to do the two steps again to get RDP to allow my remote client to connect.

Caveats/setup notes for running pytest with Jenkins on Windows 2012 R2 server

I've been writing test code in Python for the last few weeks.  Although it's been great to have Perl and Java as my predominant testing languages, it is nice to try a new scripting language on for size.

I first used Python on Linux and this last week - due to testing requirements - I needed to use it on Windows.

Python Installation :

Installing Python is pretty straightforward. Go to python.org. and download it. Installation is very easy.

After the install, I needed to add the Python to the System Path.

Pip and Pytest Installation :

Pip seemed a little different to install -vs- Linux. With Linux I could install with yum and call pip directly. To use Pip on Windows I had to call it through Python first

Installing Pytest then required me calling Python to call pip and then install PyTest. See below:

python -m pip install pytest

With Pytest installed I needed to incorporate it into my Python test class:

1) Import PyTest in the Python script using : import pytest.
2) Use an assert statement for test results.
3) Place the test statements in a test method (def testSomething(): )  For the tests to work correctly with JUnit output I needed to put the asserts in a test method.
4) Create JUnit output for Jenkins. When I did run my basic test class without outputting to an XML file it seemed to work just fine, but if I wanted to get results to Jenkins I needed to get an XML file for Jenkins to read.
5) Once I solved the above issues I needed to make sure a JUnit XML file was created. This is how I called it below:

python -m pytest scripts\CheckMSIBuildStatus.py --junitxml=results.xml

Running a java test class in a maven project on the command line

Sometimes I need to write a quick one-off Java test class against a developer's maven project or class within a maven project.

Easy to do locally on my Mac but my Mac is not a production environment.

In this case this necessitates that I run this test class on a remote machine (UNIX or Windows) where I'd rather not setup a full development environment and most likely have command line access.

Given a minimal production environment the requirements are : I can compile the developer's class, supporting classes and libraries, my test class, and then test the resulting service; in this case a small micro service the developer created.

Pre-requisites, in my case, needed on the remote machine are :

1) Java

2) GIT

3) Maven

4) If needed, the service deployed and running under systemd. 

Once I have the first three components installed, I clone the GIT repository and then try to build the maven project, usually with a mvn clean install -Dmaven.test.skip=true.

If I have any dependencies to this project, I try to use a library manager referenced in a settings.xml file under .m2 (like Artifactory) to make the project compile.

If this succeeds, I try to compile my stand-alone test class. Since I'd rather not run the developer's unit integration tests I run this command :

mvn install -DskipTests

This allows my class to compile. If this works I then I run this command :

mvn exec:java -Dexec.mainClass="com.[path to my test class].[my test class name]" -Dexec.classpathScope=test -e

Running this command allowed me to launch the Java driver program that then - in this case - created a new instance of the developer's monitor class that would run in it's own Thread through Runnable, which I could subsequently test.