The simplest resolution or explanation is usually the most common solution

In engineering, we have to diagnose and fix issues all the time.

In my case I've found that the problems I encounter when I find a build broken, a class that won't compile, or a problematic configuration that won't make a application run, is most likely due to a slight misconfiguration, misspelling, or not reading the directions thoroughly.

In other words, the easiest solution is usually the most common fix to a problem you've encountered.

Maybe this is why we have Occam's Razor as a principle?

Quick iptables setup to close off an outgoing IP for testing network interruption scenarios

I think for a lot of software testers these days, we have to test systems that span multiple servers.

Whether it's JGroups, 0MQ, or other messaging protocols, you have to make sure that systems behave correctly - or fail gracefully - when connections go down.

I posted about iptables before but it's good to review a couple of simple steps.

If I'm on a Linux system (like CentOS) I can test how a certain application might behave once it cannot connect to an external server.

If it's a simple system setup where you're testing just one application connection to another external application it's as easy as :

iptables -A OUTPUT -d [ip address] -j DROP

-A OUTPUT means to append to the OUTPUT chain (from this server to an external server)

-d is for the destination IP you want to block

-j is for jump target. If a packet matches what was stated for -A and -d then what should iptables do? In this case, I'm adding this packet to the DROP chain.

and...when I'm done testing and need to reenable communication to that IP,

iptables -F

Solving 'Plugin org.apache.maven.plugin could not be found', or 'Could not find artifact in artifactory-release'

We recently purchased an Artifactory 3.4 pro license where I work, and I've been setting it up to work with Jenkins.

Unfortunately, Jenkins was failing when it was trying to download an Apache Maven dependency with a virtual repository I'd created.

Either I'd forgotten a step or it'd been already setup when I'd been playing with the evaluation version, but I'd failed to link a huge online maven 2 repository to my virtual repo.

[NOTE: I was also trying to update the .m2/settings.xml file and that wasn't working either in Jenkins. ]

If you're getting messages in your Jenkins console output like this :

[ERROR] Plugin [...] or one of its dependencies could not be resolved: Failed to read artifact descriptor for [...]: Could not find [...] in artifactory-release

and have setup your Jenkins job (AKA job) to work with Artifactory, this might be the issue.

Once I had my virtual repository serving artifacts from my local repository, I failed with adding a remote repository - and one of the most crucial : maven2.

In my case  org.apache.maven.plugins:maven-clean-plugin:2.5, wasn't found.

I added a new remote repository pointing to : http://repo.maven.apache.org/maven2/

Then I went back to my virtual repo and added the Maven repo under Edit Virtual Repository > Basic Settings > Repositories.

I launched the build again and the error went away.

Running nmap scans to verify services aren't disrupted (e.g. elasticsearch)

I've worked in the software security space for about 5 years now, both in identity management and now SIEM/log analytics.

One useful UNIX command I never had experience with - until now - was nmap.

A big problem that we've encountered has been some services that work with elasticsearch were easily disrupted by external nmap scans.

To remedy this we had to reduce the number of externally accessible ports and also use nginx as a reverse proxy that would make people log into our web interfaces with a username and password.

NMAP:

There seem to be non-intrusive and more intrusive versions of nmap, to test what ports are open on a remote server and also more aggressive scanning and faster execution, respectively.

Some sample comands:

nmap -p 1-65535 [IP of server]
nmap -p [port range],[another individual port if needed] -T4 -A -v [IP of server]

These commands were definitely helpful when trying to verify the lock down of our ports; especially with some services like elasticsearch and cassandra. Additionally putting nginx in front of web browser services (e.g. elasticsearch HQ) that helped out even more.

nmap is certainly a nice tool for testing port lockdown.

Pesky SSH KnownHosts file (WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!)

I posted earlier in the year about SSH reverse tunneling.

At my new job, I haven't had as many issues in that department but I do re-image servers quite a bit and then need to SSH back into the machine and get those pesky errors informing me there might be a man-in-the-middle attack. Well, this is an internal firewalled server, so that's extremely unlikely.

When this happened I started off by runing ssh-keygen -R[hostname|IP address]

and that worked well, but if you by chance have more than one known_hosts file under your .ssh it might not work.

When that became a little more tedious I tried just deleting the known_hosts* file(s) under .ssh. That worked too, but it's a little too much for the task at hand. Kinda like taking a sledgehammer to a small problem.

I ultimately decided that I wanted something a little less severe that would tackle the short-term problem. The best solution is to pass command line arguments when you SSH.

ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@[hostname|IP address]

I found a few sites that mentioned this but, it was Linux Commando who provided good detail. Thanks.

Since I was also doing SSH scripting with Perl, I though I would add a step to my script to wait for the SSH daemon to start after a re-image/reboot.

Here's the code, and it was a little more difficult to get to work since I don't work with Perl as regularly (although more recently) and the Net::SSH::Perl documentation could offer a few more examples of how to setup the options.

Here's the code :

eval

                {

                        my %params;

                        $params{"strict_host_key_checking"} = "no";

                        $ssh = Net::SSH::Perl->new($passedHost, %params, options => ["UserKnownHostsFile /dev/null"] );

                        $ssh->login($user, $pass, %params);

                };

                if ($@) {

                warn "Cannot SSH yet. Here's the error message below:\n$@Waiting 30 seconds for SSH daemon to come up.\n";

                }

                sleep 30;

Setting up Windows 2012 Jenkins slave : Part 2

This is the second update on my experiences setting up a Windows 2012 Jenkins slave, where other Jenkins instances have been Unix based.

Let me just say that Jenkins isn't as easy to setup on Windows. You probably already knew that.

As an overview, this is the high-level way we write and build code and what is needed for continuous integration:

1) Java 7 and 8
2) Eclipse or IntelliJ for writing code and running tests outside the command line or Jenkins
3) Maven 3x
4) GIT 1.8.4 on Mac, GIT (msysgit 1.9.2) on Windows
5) Jenkins master running on Linux.
6) Jenkins slave agent running as a slave app on Windows
7) Cygwin for windows

Further problems encountered setting up Jenkins on Windows 2012, besides what I detailed in prior post.

1) SH scripts for running tests need to run on Windows
2) Access denied issues when running certain files (e.g BAT)
3) CreateProcess from Jenkins cannot find files to run

The first issue above is that we use SH scripts (in Bash) to setup testing and run our tests after all other packages are built. Windows won't recognize SH scripts as runnable. I decided that since I had Cygwin installed with Bash I could launch these SH scripts from within a BAT file.

The BAT file is basically one line:

C:\cygwin64\bin\bash.exe --login "C:\jenkins\workspace\...\deployScript.sh"

I originally had it start with bash, but the process couldn't be found when done through Jenkins, since it was launched through CreateProcess. I solved my 3rd issue above by fully qualifying what process I wanted to kick off under cygwin with the full path of where bash lived, and I was done with my BAT script.

The 2nd issue above was seen in build failures inside of Jenkins runs where I would see access denied. I thought I could change permissions in the POM files where these scripts were launched but that didn't work.

Ultimately, I found that access denied with my scripts was caused by Eclipse. Eclipse wouldn't make some of the new files I created executable. I clicked on properties and updated the scripts to make them fully executable, and had no further issues.

Reverse SSH tunneling to get around corporate intranet port blocking

Have a port that is blocked between two servers you use? Let's say you use ports above 1024, for example, in the 8000-9000 range for web applications or some other proprietary application you write.

Let's also say you have a Linux instance you SSH to that needs to connect back to you own laptop or another Linux machine, but the port on your local machine cannot be accessed from the remote Linux box unless you have IT open that port, which means bugging them with a ticket.

Here's something I learned the other day.

In my scenario I needed to connect to a web application, running on my local Mac Book Pro, using port 9999 when I was connected via VPN, from a Linux Openstack instance.

Unfortunately, I couldn't connect to the port with a simple program like curl since the port was blocked.

Here are the steps so I could allow SSH to serve up that port from the remote Linux instance back to my Mac.

Detailed Steps:

You need two terminal windows on the machine you need to connect back to. I had two tabs open in the Terminal window on my Mac.

1st terminal window:

     Change this string to what port you need to open >

     ssh -R 9999:localhost:9999 -l [root or whoever you connect as]@[the virtual server you connect to]

     If you don't want to use localhost, replace with server name you want to connect back to.

     You will now be on the remote linux box

2nd terminal window:

     ssh [root or whoever you connect as]@[the virtual server you connect to]

     You will now be on the remote linux box in the 2nd terminal window.

     Try using curl to test the port that you want to connect back to >

     curl https://localhost:9999//index.html --insecure

          Again, replace localhost with whatever server name you'd like to use to connect back to.