tag:blogger.com,1999:blog-21482253260899190632024-03-05T16:57:58.058-08:00Boulder Flatiron BlogAndrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.comBlogger121125tag:blogger.com,1999:blog-2148225326089919063.post-91691628100086713022020-10-12T15:36:00.008-07:002021-08-26T17:35:31.281-07:00Exploring ELK (Elastic) Stack for hack-a-thon<p></p><p>At my current gig, our group finally got to do hack-a-thon week and I joined a team project that tied together a few of the technologies I've been testing and sporadically using the last five to ten years of my career in automation and quality assurance testing. <br /></p><p></p><p></p><p>Ultimately, the purpose of this hack-a-thon project I joined was to explore the collection, pushing, indexing, and analysis/charting from logs that we collect from our own Docker swarm containers and storage nodes.<br /></p><p></p><p>It's funny that I haven't used a stack like this until now - having heard about it plenty in the past - but sometimes things come together nicely where all the technologies do mesh well; especially within the orchestration framework of Kubernetes and containerization support of Docker.</p><p>Like any stack exploration, there were some parts of ELK I hadn't really used before: Logstash and Kibana. But to have experience in other parts of the stack made the process go a little easier to get a bit more done for week of work with a POC.<br /></p><p> Technologies used:</p><ol style="text-align: left;"><li>Elasticsearch (v. 7.9.2)<br /></li><li>Logstash (v. 7.9.2)<br /></li><li>Kibana (v. 7.9.2)<br /></li><li>Java (openjdk version "1.8.0_265") <br /></li><li>Nginx (nginx version: nginx/1.14.0 (Ubuntu) )<br /></li><li>Docker (Docker version 19.03.12, build 48a66213fe)<br /></li><li>Ubuntu 18.04 (gotten from docker hub) <br /></li><li>Kubernetes (10.4.1)<br /></li><li>Artifactory (v 6.8.7)<br /></li></ol><p>I had the thought before starting with ELK that Grafana with something like Prometheus would also serve this purpose well, but ultimately my mind was changed and I was really psyched to have revisited and tied all these ideas together for something that could be matured for better log analysis and alerting.</p><p>The most rewarding parts of this project was seeing how well stuff could work in one container (not having to assign one process to an individual container - for the time being) and letting the deployed container run in a Kubernetes namespace with plenty of RAM and volume space.<br /></p><p>The biggest gotchas were with Kubernetes and making sure that I had the proper kubeconfig file, namespace created and a proper YAML file for creating a service and deployment to a node within my namespace. <br /></p><div><p> <br /></p><p><br /></p></div>Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-72076279250990644962020-06-10T10:57:00.001-07:002020-06-11T08:49:29.130-07:00Using Ansible with vCenter and SSH callsCan't believe it took me so long to explore Ansible for configuration management and automation.<br />
<br />
I heard about it back in 2014 and tested against it since we used it to install our micro services framework.
<br />
<br />
I used a good intro <a href="https://youtu.be/0_qwOKlBlo8" target="_blank">video tutorial</a> to try making a vCenter connection and listing hosts.<br />
<br />
Problems encountered were the YAML syntax, using local versus remote connections, SSH with sudo, small syntax errors and using the ansible.cfg and hosts file under /etc/ansible, which I created manually<br />
<br />
<b>Initial vCenter Tests:</b><br />
<br />
Here's some initial code for connecting to vCenter:<br />
<br />
cat auth_vcenter.yml<br />
---<br />
- hosts: localhost<br />
vars:<br />
user_readme: 'Welcome to vCenter tests'<br />
tasks:<br />
- debug:<br />
msg: "Starting test aginst vcenter"<br />
- name: Including Secret Environment Items<br />
include_vars:<br />
file: secret67.yml<br />
name: secret<br />
- name: vcenter login<br />
uri:<br />
url: "https://{{secret.vcenter}}/rest/com/vmware/cis/session"<br />
force_basic_auth: yes<br />
method: POST<br />
user: "{{secret.username}}"<br />
password: "{{secret.password}}"<br />
status_code: 200<br />
validate_certs: no<br />
register: login<br />
- name: Get hosts from vCenter<br />
uri:<br />
url: "https://{{secret.vcenter}}/rest/vcenter/host"<br />
force_basic_auth: yes<br />
validate_certs: no<br />
headers:<br />
Cookie: "{{login.set_cookie}}"<br />
register: vchosts<br />
- debug: var=login<br />
- debug: var=vchosts<br />
<br />
This works with a YAML secrets file which for initial testing is fine but passwords should be encrypted so it's not a longterm solution:<br />
<br />
cat secret67.yml<br />
---<br />
username: administrator@vsphere.local<br />
password: <your password=""></your><br />
vcenter: 10.117.180.10<br />
<div>
<br /></div>
<b>Initial SSH Tests:</b><br />
<br />
Here's some other code for trying out an automated SSH connection with some initial commands<br />
<br />
One that that got me was authentication errors so to make the login process work I had to run:<br />
<br />
ssh-copy-id -i ~/.ssh/id_rsa.pub username@10.117.155.123<br />
<br />
cat testone.yml<br />
---<br />
- hosts: 10.117.155.123<br />
vars:<br />
user_readme: 'Welcome to this machine!!!! Hey good!'<br />
tasks:<br />
- debug:<br />
msg: "Starting test against server"<br />
- name: List contents<br />
command: ls -lta<br />
register: out1<br />
- name: Touch file<br />
file:<br />
path: $HOME/test_server.txt<br />
state: touch<br />
register: out2<br />
- name: Create target directory<br />
file: path=~/testit state=directory mode=0755<br />
register: out3<br />
- name: simple file try<br />
copy:<br />
dest: ~/testit/README.txt<br />
content: " {{ user_readme }} "<br />
register: out4<br />
- debug: var=out1<br />
- debug: var=out2<br />
- debug: var=out3<br />
- debug: var=out4<br />
<div>
<br />
<b>Hosts and config Files:</b><br />
<b><br /></b>
For the SSH testing these initial files seems to work for me. Very brief and to the point.<br />
<br />
$ cat ansible.cfg<br />
[defaults]<br />
remote_user=<user ssh="" to="" use="" you=""></user><br />
<br />
$ cat hosts<br />
all:<br />
vars:<br />
ansible_ssh_user=<user ssh="" to="" use="" you=""></user><br />
ansible_ssh_pass=<clear passsword=""></clear><br />
hosts:<br />
10.117.180.10<br />
<div>
<br /></div>
<br />
<b>Summary:</b><br />
<br />
To watch the video, setup Ansible on Mac, and get these tests working should take a day or two, max, in my opinion.<br />
<br />
Next steps are to see about using Ansible for further automation testing.</div>
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-48151315936155981612019-12-03T21:01:00.003-08:002019-12-03T21:01:27.411-08:00Pycharm IDE. How to fix missing source directories in Project view. Doesn't show file or folder structure.Possibly due to a git merge between two of my computers on the same GIT project I lost the project structure.
<<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJD0Ordwy7glXswXLHUZk9VwqA7ELITOb4MBEhJ-iODp0nNSt8-N_HFFKL7q8RyKGsjDZUC1YIBRlNqMTIU5rXBf-fh9sKALeOSbCCnzQeaH2kXUdFD6omm5eZ2vpcoZSbREb0eFNbqJ4/s1600/pycharm_missing_content_root.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJD0Ordwy7glXswXLHUZk9VwqA7ELITOb4MBEhJ-iODp0nNSt8-N_HFFKL7q8RyKGsjDZUC1YIBRlNqMTIU5rXBf-fh9sKALeOSbCCnzQeaH2kXUdFD6omm5eZ2vpcoZSbREb0eFNbqJ4/s640/pycharm_missing_content_root.png" width="640" height="378" data-original-width="1600" data-original-height="946" /></a></div>
I was able to research this and re-add the content root under Preferences > Project > Project Structure. I wasn't as intuitive as I initially thought but here is the dialogue where I add it.
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxwhyphenhyphenpyIya4FCBQBnktfjFf3SPKVbrEmqZlruJdti8-P-31eOjlA7HSf4jk0kH9BBXUg-H0LYgHhOvI-76wi_EilfZewvmUEY8_ZHvrT2YMG-4OOS7SLYh6BNnIda5uEUhwWF3Ba0weFI/s1600/pycharm_project_structure_content_root.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxwhyphenhyphenpyIya4FCBQBnktfjFf3SPKVbrEmqZlruJdti8-P-31eOjlA7HSf4jk0kH9BBXUg-H0LYgHhOvI-76wi_EilfZewvmUEY8_ZHvrT2YMG-4OOS7SLYh6BNnIda5uEUhwWF3Ba0weFI/s640/pycharm_project_structure_content_root.png" width="640" height="488" data-original-width="1600" data-original-height="1220" /></a></div>
All fixed!
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-53221521819176223912019-06-18T12:43:00.003-07:002020-10-20T13:11:47.523-07:00Being more pythonic with the shortcut conditional : A if C else BI had these two blocks of "if else" code that I found too crude.<br />
<br />
<div class="code">
<b><span style="color: #2b00fe;">test_case_data = None
<br />
test_case_data2 = None
<br />
<br />
if test_already_exists(tests_object_list, test1_name) is not None:
<br />
print(">>>> Test 1 already exists. skipping the add")
<br />
else:
<br />
print(">>>> Test 1 doesn't exist. adding")
<br />
test_case_data = add_test_case(api_conn, int(project_id), test1_name)
<br />
<br />
if test_already_exists(tests_object_list, test2_name) is not None:
<br />
print(">>>> Test 2 already exists. skipping the add")
<br />
else:
<br />
print(">>>> Test 2 doesn't exist. adding")
<br />
test_case_data2 = add_test_case(api_conn, int(project_id), test2_name) </span></b></div><div class="code"> </div><p>
I took the code and applied Guido's "ternary solution," Link <a href="https://mail.python.org/pipermail/python-dev/2005-September/056846.html"> here.</a></p><p>
<br />
All of the above code became three-four lines of code and I was able to apply a one-line Pythonic truthy and falsy evaluation on the return object. Very lambda-ish. </p><p><br />
<span style="color: red;"><b>[Correction: What I had earlier with data = func_call if True else other_func_call, was incorrect. Corrected below. So it's really A if A (when A is not None) else B (which in this case creates a new test case object.]
</b></span></p><div class="code">
<span style="color: #2b00fe;"><b>for current_test in tests_lists:
<br />
test_exists_data = test_already_exists() # Note returns None or or existing test data
<br />
test_case_data = test_exists_data if test_exists_data else add_test_case()
</b></span><br />
<br />
...further code that doesn't contribute to A if C else B </div><div class="code"> </div>
As a side note: This same elegance in code I also found with the use of the "with," statement using the Context Manager.
But for this, a lot more compact, readable and Pythonic.
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-77401456396494530302019-05-21T23:19:00.002-07:002020-08-14T14:23:10.523-07:00When the default pip install won't work. Using pip directly with a github repoI work at a company where we cannot publish some (or all) Python modules to pyPI.<br />
<br />
95% of the time before last week I would use:<br />
<br />
pip/pip3 install [module name]<br />
<br />
to install modules that I needed.<br />
<br />
If that doesn't work and you have access to the git repo, another command that might be successful is installing directly from the repo. Here's an example:<br />
<br />
pip3 install git+https://github.com/[username]/repo<br />
<br />
And if you've downloaded the repo using git clone and there's a setup.py, which should be standard for a properly maintained module - you can run:<br />
<br />
pip3 install -e .<br />
<br />
Remember, sometimes you need to work around less-standard, less user-friendly modules so thankfully there are other pip options that can hopefully serve your needs.Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-50189810955326250792018-05-09T21:12:00.002-07:002018-05-10T08:23:09.961-07:00Trying out my Python 3 skills on HackerRankI've been using HackerRank to get my Python coding chops up a little better then where they've been before.<br />
<div>
<br /></div>
<div>
Tried a medium challenge on HackerRank called Piling Up</div>
<div>
<br /></div>
<div>
My solution isn't as elegant as others but it works and constantly reduces the size of the cube size list that need to be rearranged from horizontal to vertical where the block size needs to be equal or smaller than the block below it.</div>
<div>
<br />
<br /></div>
<div>
<div>
for tests in range(int(input())):</div>
<div>
cube_length = int(input())</div>
<div>
list_of_cubes = list(map(int, input().split()))</div>
<div>
<br /></div>
<div>
last_value_placed_on_stack = 0</div>
<div>
all_items_on_placed_on_stack = False</div>
<div>
cannot_stack_anymore = False</div>
<div>
<br /></div>
<div>
a = 0</div>
<div>
z = int(len(list_of_cubes)) - 1</div>
<div>
<br /></div>
<div>
while a < int(len(list_of_cubes)):</div>
<div>
if cannot_stack_anymore:</div>
<div>
break</div>
<div>
if int(len(list_of_cubes)) == 1:</div>
<div>
if int(list_of_cubes[a]) <= last_value_placed_on_stack:</div>
<div>
all_items_on_placed_on_stack = True</div>
<div>
break</div>
<div>
else:</div>
<div>
break</div>
<div>
while z > 0:</div>
<div>
if list_of_cubes[a] >= int(list_of_cubes[z]):</div>
<div>
if list_of_cubes[a] <= last_value_placed_on_stack or last_value_placed_on_stack == 0:</div>
<div>
last_value_placed_on_stack = list_of_cubes[a]</div>
<div>
list_of_cubes.pop(a)</div>
<div>
z = z - 1</div>
<div>
break</div>
<div>
else:</div>
<div>
cannot_stack_anymore = True</div>
<div>
break</div>
<div>
if list_of_cubes[z] >= int(list_of_cubes[a]):</div>
<div>
if list_of_cubes[z] <= last_value_placed_on_stack or last_value_placed_on_stack == 0:</div>
<div>
last_value_placed_on_stack = list_of_cubes[z]</div>
<div>
list_of_cubes.pop(z)</div>
<div>
z = z - 1 </div>
<div>
break</div>
<div>
else:</div>
<div>
cannot_stack_anymore = True</div>
<div>
break</div>
<div>
<br /></div>
<div>
if all_items_on_placed_on_stack:</div>
<div>
print("Yes")</div>
<div>
else:</div>
<div>
print("No")<br />
<br /></div>
</div>
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-64791180858578079422018-04-27T16:24:00.000-07:002018-04-27T16:24:07.776-07:00Computing Fibonacci in GoHad a coding whiteboard one of my first interviews back in early April.<br />
<br />
It's nice to use Go Playground to prove it out and validate it after you get home.<br />
<br />
For the whiteboard I had to do it iteratively. I added the recursive func after the fact<br />
<br />
Here's the Go Playground link: https://play.golang.org/p/KDQaQlq8ONl<br />
<br />
package main<br />
<br />
import (<br />
<span style="white-space: pre;"> </span>"fmt"<br />
)<br />
<br />
func main() {<br />
<span style="white-space: pre;"> </span>fmt.Println("Hello, playground. Test for computing Fibonacci numbers")<br />
<br />
<span style="white-space: pre;"> </span>computeFibonacci(10)<br />
<span style="white-space: pre;"> </span>computeFibonacciRecursively(10, 0, 1)<br />
}<br />
<br />
func computeFibonacci(numberOfNumbers int) {<br />
<br />
<span style="white-space: pre;"> </span>penultimate := 0<br />
<span style="white-space: pre;"> </span>ultimate := 1<br />
<br />
<span style="white-space: pre;"> </span>fmt.Printf("\nNext number (iterative): %d", penultimate)<br />
<span style="white-space: pre;"> </span>fmt.Printf("\nNext number (iterative): %d", ultimate)<br />
<br />
<span style="white-space: pre;"> </span>for i := 2; i < numberOfNumbers; i++ {<br />
<br />
<span style="white-space: pre;"> </span>currentSum := penultimate + ultimate<br />
<span style="white-space: pre;"> </span>fmt.Printf("\nNext number (iterative): %d", currentSum)<br />
<span style="white-space: pre;"> </span>penultimate = ultimate<br />
<span style="white-space: pre;"> </span>ultimate = currentSum<br />
<br />
<span style="white-space: pre;"> </span>}<br />
<br />
}<br />
<br />
func computeFibonacciRecursively(numberOfNumbers int, penultimate int, ultimate int) {<br />
<br />
<span style="white-space: pre;"> </span>if numberOfNumbers == 0 {<br />
<span style="white-space: pre;"> </span>return<br />
<span style="white-space: pre;"> </span>} else {<br />
<span style="white-space: pre;"> </span>fmt.Printf("\nNext number (recurse method): %d", penultimate)<br />
<span style="white-space: pre;"> </span>numberOfNumbers--<br />
<span style="white-space: pre;"> </span>computeFibonacciRecursively(numberOfNumbers, ultimate, penultimate+ultimate)<br />
<br />
<span style="white-space: pre;"> </span>}<br />
<br />
}<br />
<div>
<br /></div>
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-22914143640757177462017-02-03T13:37:00.002-08:002017-03-30T15:56:49.735-07:00Powershell problem : With remote PSSession, Running EXE in path with spaces and getting back response using Write-OutputI think some of the most trouble I've recently had in coding Powershell has been trying to run an executable on a remote Hyper-V VM that was located under Program Files and then actually getting the response back properly so I could iterate though the results.<br />
<br />
The first trickiness was with running the command properly when there were spaces in the path.<br />
<br />
In double quotes you have to use the ampersand (&) and then write the path to the EXE in single quotes and then at the end of the string before the closing quotes character, append an argument.<br />
<br />
For example :<br />
<br />
$commandLocation = "& 'C:\Program Files\....\application_name.exe' argument"<br />
<br />
This is the proper syntax that you can then pass to an Invoke-Expression that's inside a ScriptBlock called by Invoke-Command. Invoke-Expression is a very helpful commandlet to run EXEs, when, for example, running Powershell on a remote machine.<br />
<br />
The second issue was understanding how I could use Write-Output to pipe the remote call output back a Powershell variable within the ScriptBlock that I could then use in the calling function.<br />
<br />
I read that you could use 4>&1 at the end of the ScriptBlock but never found it useful in my case.<br />
<br />
In my case, I just had a ScriptBlock assigned to a variable and then ran the ScriptBlock using an Invoke-Command on a separate line.<br />
<br />
There was some funkiness with WriteHost appending to the invoke-command so you have to disable that and then return the pure unadulterated Invoke-Command Result that I assigned to a variable.<br />
<br />
Then I could iterate over the lines in the method that called the method below.<br />
<br />
Here's the example code :<br />
<br />
<b>Function RunCommandOnRemoteMachine([string] $blahParameter, [string] $remoteMachineAddress)</b><br />
<b>{</b><br />
<b><br /></b>
<b> $defaultBlahCommandLocation = "& 'C:\Program Files\...\blah.exe' "</b><br />
<b><br /></b>
<b> WriteLogAndConsole "Default blah directory : $defaultBlahCommandLocation"</b><br />
<b> </b><br />
<b> $fullCommand = $defaultBlahCommandLocation + $blahParameter</b><br />
<b><br /></b>
<b> WriteLogAndConsole "Full command with arguments being run : $fullCommand"</b><br />
<b><br /></b>
<b> $scriptToExecute = { param($passedCommand)</b><br />
<b> begin {</b><br />
<b> Write-Host "Start of command:"</b><br />
<b> }</b><br />
<b> process {</b><br />
<b> $consoleOutput = Invoke-Expression -Verbose -Command $passedCommand</b><br />
<b> }</b><br />
<b> end {</b><br />
<b> Write-Host "Output from remote command:"</b><br />
<b> Write-Host $consoleOutput</b><br />
<b> Write-Host "************"</b><br />
<b> Write-Output $consoleOutput</b><br />
<b> }</b><br />
<b> }</b><br />
<b><br /></b>
<b> $blahCommandResult = Invoke-Command -Session $global:session -Verbose -ScriptBlock $scriptToExecute -ArgumentList $fullCommand</b><br />
<b><br /></b>
<b> #Write-Host "Result of blah argument command: " $blahCommandResult</b><br />
<b><br /></b>
<b> return $blahCommandResult</b><br />
<b><br /></b>
<b>}</b>Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-26680698174515087882017-01-19T17:10:00.000-08:002017-01-19T17:10:10.821-08:00Powershell New-PS Drive using Administrator credentials. User name or password is incorrectSometimes you forget the obvious when your doing scripting.<br />
<br />
Although not recommended, I was using powershell to create a new psdrive session using the Administrator.<br />
<br />
Problem is that with just Administrator and the password, powershell was complaining about a bad username and password combination.<br />
<br />
Since the VM was connected to a development domain I wasn't qualifying the Administrator name therefore confusing powershell.<br />
<br />
I add the computername\Administrator and tried again. Much better successAndrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-24613695133620179972016-12-21T12:48:00.000-08:002017-01-19T16:58:58.683-08:00Elasticsearch analogiesI work a lot with Elasticsearch.<br />
<br />
At my job we have migrated from using SQL Server to Elasticsearch for log storage.<br />
<br />
Sometimes, when you're learning new grammar for a new technology you're working on it's nice to have metaphors or similes to use when learning the component parts.<br />
<br />
One of the best similies I've heard of for what Elasticsearch represents is that it's like an index at the end of a book. Additionally, Elasticsearch by many, even those who work at Elastic, consider it not a DB.<br />
<br />
If you start using Elasticsearch a lot more you'll begin to work with types, templates and mappings.<br />
<br />
A type could be analogous to a regular SQL table.<br />
<br />
Mappings could be similar to a a table column that stores a field a certain way that could be something like a string, int, varchar.<br />
<br />
How mappings and settings are created and loaded has changed from Elasticsearch 1.x to 2.x+. Now that we use Elasticsearch 2.x we have to load template files that are used when new indices are created. They combine settings and the mappings for the fields that get parsed out and placed into an index.<br />
<br />
Thanks to this website for giving me those Aha! analogies.<br />
<br />
http://swops.com/blog/elasticsearch-mappings-and-templates<br />
<br />Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-86996589679886191302016-12-21T12:46:00.001-08:002016-12-26T16:59:42.375-08:00Rebuild project of 18 year old Mansfield ToiletI just completed a toilet rebuild last weekend. The toilet began leaking a year or two ago; especially during warm weather.<br />
<br />
Although I wasn't working the whole time the total time from start to finish was a half day. I did initial work and investigations starting Friday night, I let the tank dry out overnight, and I finished about 1 PM the following day having started around 10 AM with the installation of the new parts on the tank.<br />
<br />
Actual time working was probably around 4 hours.<br />
<br />
Work entailed replacing the fill valve, flush valve, new bolts attaching the tank to the bowl, new shutoff valve and supply line too.<br />
<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEin4M2MeSgATtqvLrWpd4RZTy4zTqjQOsZss-Bt7HmrtdHjpiaB5bVUFyIAjkFuLGz4PJBD7FoOGwXXpiL1FEot_aqpOaCTdTRE4n_zh_H3w5qBOCEeLq1i8-yw0KSM65HUVv7zi1zPtFA/s1600/IMG_0695.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEin4M2MeSgATtqvLrWpd4RZTy4zTqjQOsZss-Bt7HmrtdHjpiaB5bVUFyIAjkFuLGz4PJBD7FoOGwXXpiL1FEot_aqpOaCTdTRE4n_zh_H3w5qBOCEeLq1i8-yw0KSM65HUVv7zi1zPtFA/s400/IMG_0695.JPG" width="400" /></a></div>
<br />
<div style="text-align: center;">
New shutoff valve</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMhyzarRXvPZLjBIdp1ZcXOQfvPENU66uDj9498tD6tZyegZeXy6yeHkb0KDxhee6kmRUpqj3o27DYYoKxe1Xym85OyWRL4RFka_NWEFkN7jGCBfc6plpACZWStYUZV_soM47VjVTU2oc/s1600/IMG_0700.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMhyzarRXvPZLjBIdp1ZcXOQfvPENU66uDj9498tD6tZyegZeXy6yeHkb0KDxhee6kmRUpqj3o27DYYoKxe1Xym85OyWRL4RFka_NWEFkN7jGCBfc6plpACZWStYUZV_soM47VjVTU2oc/s400/IMG_0700.JPG" width="400" /></a></div>
<div style="text-align: center;">
New Supply Line to Tank</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNUe2-72OvVmT68qxnNhhby7fOsTFTbIYRHJOJZWyD9LFzhA2E8WK-qV2iR8B8Vzu1BLVIFVYQ5BtAYzn1A5rsS7N-Dp9hCI8TxxYF7mRJmZDtTtxqb0hqZ5HoQrcg4O31sOw46kUVDBk/s1600/FullSizeRender.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNUe2-72OvVmT68qxnNhhby7fOsTFTbIYRHJOJZWyD9LFzhA2E8WK-qV2iR8B8Vzu1BLVIFVYQ5BtAYzn1A5rsS7N-Dp9hCI8TxxYF7mRJmZDtTtxqb0hqZ5HoQrcg4O31sOw46kUVDBk/s400/FullSizeRender.jpg" width="400" /></a></div>
<div style="text-align: center;">
All parts replaced in dry tank</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_xeLF0cUU5aU2hW2Xh9CSg9_6UhT6dIv-q08VKRBob5XFd18dhu2Wsx3ckxjPxigS72MalLxcHfnaiv0ZVxv9r9NaibAfYCaLjQX757AC6-VHPepWozgpcqJl-He-VRHi_BO2-0uNa8Q/s1600/IMG_0701.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_xeLF0cUU5aU2hW2Xh9CSg9_6UhT6dIv-q08VKRBob5XFd18dhu2Wsx3ckxjPxigS72MalLxcHfnaiv0ZVxv9r9NaibAfYCaLjQX757AC6-VHPepWozgpcqJl-He-VRHi_BO2-0uNa8Q/s400/IMG_0701.JPG" width="400" /></a></div>
<div style="text-align: center;">
Tank full with water</div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
All parts were ordered from Mansfield and the cost was just under 50 bucks. I went to Home Depot for the new shutoff valve and supply line.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Total cost was about 60 bucks.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Issues encountered after the replacement was slight leaking from one of the bolts, which I had to tighten down more to prevent further leakage.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
As mentioned on my Twitter post, I'd say this home project was a 2-3 out of 10.</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<br />Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-73758112754152082162016-07-20T17:58:00.000-07:002018-12-12T21:47:13.693-08:00Perl : subroutine redefined, prototype mismatchI was refactoring my code and placing some common methods into a base module in my Perl lib directory.<br />
<br />
Unfortunately some of my new method names had name collisions with possible subroutines in other packages.<br />
<br />
I was able to rename the methods that I was exporting for use in my PL files and the errors went away.Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-52136149606280948962016-03-30T09:01:00.001-07:002016-07-21T08:52:48.206-07:00Python Indentation ErrorI've only been using Python for a year.<br />
<div>
<br /></div>
<div>
Before that, my focuses were Java and Perl.</div>
<div>
<br /></div>
<div>
An interesting error I came across, since Python has stricter criteria for formatting was: "IndentationError: expected an indented block"</div>
<div>
<br /></div>
<div>
The code in question:</div>
<div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilq5HLYKzdr6lD2Oq1oEvjFeXArHzuatHs49BplHne_B9_6yCuSsCxXC31itzmv4rV85yDfliOldf4QO3IdgwJI2Wq5Z7TzosYtDsu7LxxY36O2M_nIpRM3Xln7yU0XYC7L9K_xIT-k9o/s1600/No_logging_statement.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="60" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilq5HLYKzdr6lD2Oq1oEvjFeXArHzuatHs49BplHne_B9_6yCuSsCxXC31itzmv4rV85yDfliOldf4QO3IdgwJI2Wq5Z7TzosYtDsu7LxxY36O2M_nIpRM3Xln7yU0XYC7L9K_xIT-k9o/s400/No_logging_statement.png" width="400" /></a></div>
<br /></div>
<div>
<br /></div>
<div>
<br />
<br />
<br />
The error might be considered a red herring since the problem with the else: statement was that there was no real code to run after the if statement.</div>
<div>
<br /></div>
<div>
Once I added a logging statement:</div>
<div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiENXGJXelAwgepVW5zalQi84SCPYASGNdeyTVr0BZAJqj2qHYz-nNyWDeaCO7YXehJR2Y3-RggSoavd_yfEc1zrLVG2H-m_FJqEs-hB69drdeN_jSpXf6UnK64ZnzRnaNEmsjPy29-ZLE/s1600/Added_logging_statement.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="70" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiENXGJXelAwgepVW5zalQi84SCPYASGNdeyTVr0BZAJqj2qHYz-nNyWDeaCO7YXehJR2Y3-RggSoavd_yfEc1zrLVG2H-m_FJqEs-hB69drdeN_jSpXf6UnK64ZnzRnaNEmsjPy29-ZLE/s400/Added_logging_statement.png" width="400" /></a></div>
<br /></div>
<div>
<br /></div>
<div>
<br />
<br />
<br />
The error went away.</div>
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-59520525746689460272015-06-12T14:08:00.001-07:002017-02-10T09:35:27.518-08:00Setting up a new hyper-v Windows 2012 server to be pingable and allow RDP connectionsAt my current company we are needing to test deployments not only to Linux but Windows too.<br />
<div>
<br /></div>
<div>
To quickly have preconfigured systems come up we use virtualization (Hyper-V) along with other technologies like Vagrant and Ansible with other configuration and testing scripts like Python and/or Powershell.</div>
<div>
<br /></div>
<div>
I just setup a new Windows 2012 R2 Machine in Hyper-V using a ISO image.</div>
<div>
<br /></div>
<div>
By default, some features aren't enabled to make my testing easier. For example, to be able to ping the machine or RDP into it for GUI related tasks.</div>
<div>
<br /></div>
<div>
To enable Windows to respond to a ping you have to go into Windows Firewall with Advanced Security > Inbound Rules and enable File and Printer Sharing (Echo Request - ICMPv4-In) by right clicking on the rule name.</div>
<div>
<br /></div>
<div>
For remote desktop connections I had to do two things:</div>
<div>
<br /></div>
<div>
1) Type SystemPropertiesRemote and select "Allow remote connections to this computer." I also deselected the "Allow connections only..." beneath it</div>
<div>
2) I ran this command: netsh advfirewall firewall set rule group="Remote Desktop" new enable=yes</div>
<div>
<br /></div>
<div>
If for some reason I disabled Allow remote connections again I would have to do the two steps again to get RDP to allow my remote client to connect.</div>
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-40316684327518978542015-05-08T17:32:00.001-07:002017-02-10T09:48:56.541-08:00Caveats/setup notes for running pytest with Jenkins on Windows 2012 R2 serverI've been writing test code in Python for the last few weeks. Although it's been great to have Perl and Java as my predominant testing languages, it is nice to try a new scripting language on for size.<br />
<br />
I first used Python on Linux and this last week - due to testing requirements - I needed to use it on Windows.<br />
<br />
Python Installation :<br />
<br />
Installing Python is pretty straightforward. Go to python.org. and download it. Installation is very easy.<br />
<br />
After the install, I needed to add the Python to the System Path.<br />
<br />
Pip and Pytest Installation :<br />
<br />
Pip seemed a little different to install -vs- Linux. With Linux I could install with yum and call pip directly. To use Pip on Windows I had to call it through Python first<br />
<br />
Installing Pytest then required me calling Python to call pip and then install PyTest. See below:<br />
<br />
python -m pip install pytest<br />
<br />
With Pytest installed I needed to incorporate it into my Python test class:<br />
<br />
1) Import PyTest in the Python script using : import pytest.<br />
2) Use an assert statement for test results.<br />
3) Place the test statements in a test method (def testSomething(): ) For the tests to work correctly with JUnit output I needed to put the asserts in a test method.<br />
4) Create JUnit output for Jenkins. When I did run my basic test class without outputting to an XML file it seemed to work just fine, but if I wanted to get results to Jenkins I needed to get an XML file for Jenkins to read.<br />
5) Once I solved the above issues I needed to make sure a JUnit XML file was created. This is how I called it below:<br />
<br />
python -m pytest scripts\CheckMSIBuildStatus.py --junitxml=results.xmlAndrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com1tag:blogger.com,1999:blog-2148225326089919063.post-3671878942472653112015-05-05T17:46:00.003-07:002015-05-11T22:29:21.872-07:00Running a java test class in a maven project on the command lineSometimes I need to write a quick one-off Java test class against a developer's maven project or class within a maven project.<br />
<div>
<br /></div>
<div>
Easy to do locally on my Mac but my Mac is not a production environment.<br />
<br />
In this case this necessitates that I run this test class on a remote machine (UNIX or Windows) where I'd rather not setup a full development environment and most likely have command line access.<br />
<br />
Given a minimal production environment the requirements are : I can compile the developer's class, supporting classes and libraries, my test class, and then test the resulting service; in this case a small micro service the developer created.</div>
<div>
<br /></div>
<div>
Pre-requisites, in my case, needed on the remote machine are :</div>
<div>
<br /></div>
<div>
1) Java</div>
<div>
2) GIT</div>
<div>
3) Maven</div>
<div>
4) If needed, the service deployed and running under systemd. </div>
<div>
<br /></div>
<div>
Once I have the first three components installed, I clone the GIT repository and then try to build the maven project, usually with a mvn clean install -Dmaven.test.skip=true.<br />
<br />
If I have any dependencies to this project, I try to use a library manager referenced in a settings.xml file under .m2 (like Artifactory) to make the project compile.</div>
<div>
<br /></div>
<div>
If this succeeds, I try to compile my stand-alone test class. Since I'd rather not run the developer's unit integration tests I run this command :</div>
<div>
<br /></div>
<div>
<div class="p1">
mvn install -DskipTests<br />
<br />
This allows my class to compile. If this works I then I run this command :</div>
<div class="p1">
<br /></div>
<div class="p1">
</div>
<div class="p1">
mvn exec:java -Dexec.mainClass="com.[path to my test class].[my test class name]" -Dexec.classpathScope=test -e</div>
<div class="p1">
<br /></div>
<div class="p1">
Running this command allowed me to launch the Java driver program that then - in this case - created a new instance of the developer's monitor class that would run in it's own Thread through Runnable, which I could subsequently test.</div>
</div>
<div>
<br /></div>
<div>
<br /></div>
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-78647405866541089942015-04-13T17:12:00.000-07:002015-04-14T11:21:09.861-07:00GIT : getting the lastest changes from your upstream master when you have no changes in your repoI clone a lot of our micro service repositories locally into my own branch, but then really don't add to them. A few months pass and then I need to re-sync.<br />
<br />
GIT seems to make it a little harder to do it compared, to say, Perforce, another source control program. At a high level you have to get the latest changes from your upstream master, that you forked from earlier, fetch and then merge.<br />
<br />
I've detailed the more individual steps below, and what I encountered. To start, you may not even have your upstream master defined!<br />
<div>
<br /></div>
<div>
<u><b>Here are the high level command line steps I took on my Mac Book Pro.</b></u></div>
<div>
<br /></div>
<div>
<div class="p1">
git fetch upstream</div>
<div class="p1">
<br /></div>
<div class="p1">
I got > fatal: 'upstream' does not appear to be a git repository. So here upstream is not defined.</div>
<div class="p1">
<br /></div>
<div class="p1">
git remote -v</div>
<div class="p1">
<br /></div>
<div class="p1">
With this command I noted that no upstream was listed. Guess I have to create it then.</div>
<div class="p1">
<br /></div>
<div class="p1">
git remote add upstream http://github[remainder of URI path]<path repository="" the="" to="">/[repository name]<repository name="">.git</repository></path></div>
<div class="p1">
<br /></div>
<div class="p1">
This command allowed me to define my upstream </div>
<div class="p1">
<br /></div>
<div class="p1">
git remote -v</div>
<div class="p1">
<br /></div>
<div class="p1">
Verification step that upstream remote was now defined</div>
<div class="p1">
<br /></div>
<div class="p1">
git fetch upstream</div>
<div class="p1">
<br /></div>
<div class="p1">
This got me the latest changes, but stored it locally in a repo called upstream/master.</div>
<div class="p1">
<br /></div>
<div class="p1">
git log</div>
<div class="p1">
<br /></div>
<div class="p1">
This verified that the latest changes - in my local master repo - were still from a few months ago so a couple more steps needed.</div>
<div class="p1">
<br /></div>
<div class="p1">
git checkout master</div>
<div class="p1">
<br /></div>
<div class="p1">
Although not needed in my case, just a verification I was on the right branch. This command will move you to the master branch in case you might be on another branch.</div>
<div class="p1">
<br /></div>
<div class="p1">
git merge upstream/master</div>
<div class="p1">
<br /></div>
<div class="p1">
This command finally synced my local unchanged branch to the latest changes in the HEAD, aka upstream.</div>
<div class="p1">
<br /></div>
<div class="p1">
git log</div>
<div class="p1">
<br /></div>
<div class="p1">
With git log I saw the latest changes were from today.<br />
<br />
<br />
Hope this helps.</div>
</div>
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-35910968960383704072015-02-15T22:32:00.000-08:002015-04-05T16:30:52.351-07:00Pad Kee Mao GaiI created a separate cooking blog...<br />
<br />
The blog writeup will move permanently here ><br />
<br />
<a href="http://andresfernandezcooking.blogspot.com/2015/03/pad-kee-mao-gai.html" target="_blank">http://andresfernandezcooking.blogspot.com/</a><br />
<br />
<br />
*************<br />
<br />
<br />
<br />
I've been cooking Thai cuisine the last 2 years and have learned quite a few dishes.<br />
<br />
I've done Lad Na, Pad Thai, Masaman, Pad Prik Khing, and other recipes, but wanted to try something new with Thai basil.<br />
<br />
I order a lot from importfood.com, and wanted to use their recipe on the website but did a little more searching to see what the recipe would look like and see the possible variations.<br />
<br />
I tried this recipe here : <a href="http://aloshaskitchen.blogspot.com/2010/07/ive-already-mentioned-my-love.html" target="_blank">Alosha's Kitchen</a> and with a couple of small modifications, I thought it was a very good recipe.<br />
<br />
To keep the heat down to medium, I used only three Thai chilies along with 12 ounces of wide noodles and a pound of chicken. I did use a whole small red onion and a red and yellow pepper; so twice the amount of onion.<br />
<br />
If you're using 12 ounces of wide noodles I would recommend a 1/4 cup of fish sauce, slightly a little less of the golden mountain and thick soy sauce and directly mix it with the chicken before putting it in the pan. I also mix my palm sugar with water for easy use and store it in the fridge so I can just pour it on the dish as I'm creating it.<br />
<br />
Here's a photo of the finished dish :<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1-juFcyQB6JrYIehGe2lIDzv9Oed1vfSw5tnYkCJvN-sbIbhJNkeToza4WTyxmPfyiDb_YXc8IbXwLvspkr__AKim4J4Y7y-iaTiRspiN1E5Gah4O7vx0sZNNaJDYdB5t2ARs-TeQv-s/s1600/pad_kee_mao_gai.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1-juFcyQB6JrYIehGe2lIDzv9Oed1vfSw5tnYkCJvN-sbIbhJNkeToza4WTyxmPfyiDb_YXc8IbXwLvspkr__AKim4J4Y7y-iaTiRspiN1E5Gah4O7vx0sZNNaJDYdB5t2ARs-TeQv-s/s1600/pad_kee_mao_gai.JPG" height="240" width="320" /></a></div>
<br />Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com1tag:blogger.com,1999:blog-2148225326089919063.post-35476640090762551292015-02-11T16:07:00.000-08:002015-02-11T16:07:08.064-08:00Comparing UNIX networking commands : netstat, lsof and ssI've realized recently there's a lot more availability in UNIX commands to look at your network (open ports, etc...).<br />
<br />
Some sample commands that do the same thing, which I'll investigate later :<br />
<br />
<br />
<div class="p1">
netstat -a -p | grep 13100</div>
<div class="p1">
<br /></div>
<div class="p1">
</div>
<div class="p1">
ss -all | grep 13100</div>
<div class="p1">
<br /></div>
<div class="p1">
</div>
<div class="p1">
lsof -n -i4TCP | grep 13100</div>
<div class="p1">
<br /></div>
<div class="p1">
These three commands do a lot of similar things, albeit with some minor tweaks; and then pipe it to grep to look at a specific port.</div>
<div class="p1">
<br /></div>
<div class="p1">
Currently, my favorite for information and formatting is lsof.</div>
<div class="p1">
<br /></div>
<div class="p1">
I'll write more later...</div>
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-66991754297673548072015-02-09T17:41:00.003-08:002015-02-09T17:41:35.407-08:00The simplest resolution or explanation is usually the most common solutionIn engineering, we have to diagnose and fix issues all the time.<br />
<br />
In my case I've found that the problems I encounter when I find a build broken, a class that won't compile, or a problematic configuration that won't make a application run, is most likely due to a slight misconfiguration, misspelling, or not reading the directions thoroughly.<br />
<br />
In other words, the easiest solution is usually the most common fix to a problem you've encountered.<br />
<br />
Maybe this is why we have Occam's Razor as a principle?Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-5828879635689766582015-01-18T21:33:00.000-08:002015-01-22T15:37:53.523-08:00Quick iptables setup to close off an outgoing IP for testing network interruption scenariosI think for a lot of software testers these days, we have to test systems that span multiple servers.<br />
<br />
Whether it's JGroups, 0MQ, or other messaging protocols, you have to make sure that systems behave correctly - or fail gracefully - when connections go down.<br />
<br />
I posted about iptables before but it's good to review a couple of simple steps.<br />
<br />
If I'm on a Linux system (like CentOS) I can test how a certain application might behave once it cannot connect to an external server.<br />
<br />
If it's a simple system setup where you're testing just one application connection to another external application it's as easy as :<br />
<br />
iptables -A OUTPUT -d [ip address] -j DROP<br />
<br />
-A OUTPUT means to append to the OUTPUT chain (from this server to an external server)<br />
<br />
-d is for the destination IP you want to block<br />
<br />
-j is for jump target. If a packet matches what was stated for -A and -d then what should iptables do? In this case, I'm adding this packet to the DROP chain.<br />
<br />
and...when I'm done testing and need to reenable communication to that IP,<br />
<br />
iptables -FAndrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-35802818780758836632014-12-29T15:08:00.001-08:002014-12-30T16:53:39.441-08:00Solving 'Plugin org.apache.maven.plugin could not be found', or 'Could not find artifact in artifactory-release'We recently purchased an Artifactory 3.4 pro license where I work, and I've been setting it up to work with Jenkins.<br />
<div>
<br /></div>
<div>
Unfortunately, Jenkins was failing when it was trying to download an Apache Maven dependency with a virtual repository I'd created.</div>
<div>
<br /></div>
<div>
Either I'd forgotten a step or it'd been already setup when I'd been playing with the evaluation version, but I'd failed to link a huge online maven 2 repository to my virtual repo.</div>
<div>
<br /></div>
<div>
[NOTE: I was also trying to update the .m2/settings.xml file and that wasn't working either in Jenkins. ]</div>
<div>
<br /></div>
<div>
If you're getting messages in your Jenkins console output like this :</div>
<div>
<br /></div>
<div>
<div class="p1">
<span class="s1"><b>[ERROR] Plugin [...] or one of its dependencies could not be resolved: Failed to read artifact descriptor for [...]: Could not find [...] in artifactory-release</b></span></div>
<div class="p1">
<span class="s1"><b><br /></b></span></div>
<div class="p1">
and have setup your Jenkins job (AKA job) to work with Artifactory, this might be the issue.</div>
<div class="p1">
<br /></div>
<div class="p1">
Once I had my virtual repository serving artifacts from my local repository, I failed with adding a remote repository - and one of the most crucial : maven2.</div>
<div class="p1">
<br /></div>
<div class="p1">
In my case <b> org.apache.maven.plugins:maven-clean-plugin:2.5, </b>wasn't found.</div>
<div class="p1">
<br /></div>
<div class="p1">
I added a new remote repository pointing to : http://repo.maven.apache.org/maven2/</div>
<div class="p1">
<br /></div>
<div class="p1">
Then I went back to my virtual repo and added the Maven repo under Edit Virtual Repository > Basic Settings > Repositories.</div>
<div class="p1">
<br /></div>
<div class="p1">
I launched the build again and the error went away.</div>
</div>
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-65482692074168427262014-12-19T15:46:00.003-08:002014-12-22T10:40:01.973-08:00Running nmap scans to verify services aren't disrupted (e.g. elasticsearch)I've worked in the software security space for about 5 years now, both in identity management and now SIEM/log analytics.<br />
<br />
One useful UNIX command I never had experience with - until now - was nmap.<br />
<br />
A big problem that we've encountered has been some services that work with elasticsearch were easily disrupted by external nmap scans.<br />
<br />
To remedy this we had to reduce the number of externally accessible ports and also use nginx as a reverse proxy that would make people log into our web interfaces with a username and password.<br />
<br />
NMAP:<br />
<br />
There seem to be non-intrusive and more intrusive versions of nmap, to test what ports are open on a remote server and also more aggressive scanning and faster execution, respectively.<br />
<br />
Some sample comands:<br />
<br />
nmap -p 1-65535 [IP of server]<br />
nmap -p [port range],[another individual port if needed] -T4 -A -v [IP of server]<br />
<br />
These commands were definitely helpful when trying to verify the lock down of our ports; especially with some services like elasticsearch and cassandra. Additionally putting nginx in front of web browser services (e.g. elasticsearch HQ) that helped out even more.<br />
<br />
nmap is certainly a nice tool for testing port lockdown.Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-90044944210483974532014-11-20T15:10:00.003-08:002014-11-20T15:13:46.068-08:00Pesky SSH KnownHosts file (WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!)I posted earlier in the year about SSH reverse tunneling.<br />
<div>
<br /></div>
<div>
At my new job, I haven't had as many issues in that department but I do re-image servers quite a bit and then need to SSH back into the machine and get those pesky errors informing me there might be a man-in-the-middle attack. Well, this is an internal firewalled server, so that's extremely unlikely.</div>
<div>
<br /></div>
<div>
When this happened I started off by runing ssh-keygen -R[hostname|IP address]</div>
<div>
<br /></div>
<div>
and that worked well, but if you by chance have more than one known_hosts file under your .ssh it might not work.</div>
<div>
<br /></div>
<div>
When that became a little more tedious I tried just deleting the known_hosts* file(s) under .ssh. That worked too, but it's a little too much for the task at hand. Kinda like taking a sledgehammer to a small problem.</div>
<div>
<br /></div>
<div>
I ultimately decided that I wanted something a little less severe that would tackle the short-term problem. The best solution is to pass command line arguments when you SSH.</div>
<div>
<br /></div>
<div>
<div class="p1">
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@<hostname address=""><ip ostname=""></ip></hostname>[hostname|IP address]</div>
<div class="p1">
<br /></div>
<div class="p1">
I found a few sites that mentioned this but, it was <a href="http://linuxcommando.blogspot.com/2008/10/how-to-disable-ssh-host-key-checking.html" target="_blank">Linux Commando</a> who provided good detail. Thanks.</div>
<div class="p1">
<br /></div>
<div class="p1">
Since I was also doing SSH scripting with Perl, I though I would add a step to my script to wait for the SSH daemon to start after a re-image/reboot.</div>
<div class="p1">
<br /></div>
<div class="p1">
Here's the code, and it was a little more difficult to get to work since I don't work with Perl as regularly (although more recently) and the Net::SSH::Perl documentation could offer a few more examples of how to setup the options.</div>
<div class="p1">
<br /></div>
<div class="p1">
Here's the code :</div>
<div class="p1">
<br /></div>
<div class="p1">
eval</div>
<div class="p1">
{</div>
<div class="p1">
my %params;</div>
<div class="p1">
$params{"strict_host_key_checking"} = "no";</div>
<div class="p1">
$ssh = Net::SSH::Perl->new($passedHost, %params, options => ["UserKnownHostsFile /dev/null"] );</div>
<div class="p1">
$ssh->login($user, $pass, %params);</div>
<div class="p1">
};</div>
<div class="p1">
if ($@) {</div>
<div class="p1">
warn "Cannot SSH yet. Here's the error message below:\n$@Waiting 30 seconds for SSH daemon to come up.\n";</div>
<div class="p1">
}</div>
<div class="p1">
</div>
<div class="p1">
sleep 30;</div>
<div class="p1">
<br /></div>
<div class="p1">
<br /></div>
</div>
<div>
<br /></div>
<div>
<br /></div>
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0tag:blogger.com,1999:blog-2148225326089919063.post-91567237239884419612014-06-18T15:43:00.001-07:002017-01-18T12:25:23.990-08:00Setting up Windows 2012 Jenkins slave : Part 2This is the second update on my experiences setting up a Windows 2012 Jenkins slave, where other Jenkins instances have been Unix based.<br />
<br />
Let me just say that Jenkins isn't as easy to setup on Windows. You probably already knew that.<br />
<br />
As an overview, this is the high-level way we write and build code and what is needed for continuous integration:<br />
<br />
1) Java 7 and 8<br />
2) Eclipse or IntelliJ for writing code and running tests outside the command line or Jenkins<br />
3) Maven 3x<br />
4) GIT 1.8.4 on Mac, GIT (msysgit 1.9.2) on Windows<br />
5) Jenkins master running on Linux.<br />
6) Jenkins slave agent running as a slave app on Windows<br />
7) Cygwin for windows<br />
<br />
Further problems encountered setting up Jenkins on Windows 2012, besides what I detailed in prior post.<br />
<br />
1) SH scripts for running tests need to run on Windows<br />
2) Access denied issues when running certain files (e.g BAT)<br />
3) CreateProcess from Jenkins cannot find files to run<br />
<br />
The first issue above is that we use SH scripts (in Bash) to setup testing and run our tests after all other packages are built. Windows won't recognize SH scripts as runnable. I decided that since I had Cygwin installed with Bash I could launch these SH scripts from within a BAT file.<br />
<br />
The BAT file is basically one line:<br />
<br />
<br />
<div class="p1">
C:\cygwin64\bin\bash.exe --login "C:\<span class="s1">jenkins</span>\workspace\...\deployScript.sh"</div>
<div class="p1">
<br /></div>
<div class="p1">
I originally had it start with bash, but the process couldn't be found when done through Jenkins, since it was launched through CreateProcess. I solved my 3rd issue above by fully qualifying what process I wanted to kick off under cygwin with the full path of where bash lived, and I was done with my BAT script.</div>
<div class="p1">
<br /></div>
<div class="p1">
The 2nd issue above was seen in build failures inside of Jenkins runs where I would see access denied. I thought I could change permissions in the POM files where these scripts were launched but that didn't work.</div>
<div class="p1">
<br /></div>
<div class="p1">
Ultimately, I found that access denied with my scripts was caused by Eclipse. Eclipse wouldn't make some of the new files I created executable. I clicked on properties and updated the scripts to make them fully executable, and had no further issues.</div>
Andrew Fernandezhttp://www.blogger.com/profile/17860091392677634852noreply@blogger.com0