tag:blogger.com,1999:blog-52771610376042600372024-03-18T20:22:02.888-07:00Expect LiteAutomation for the rest of usCraighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.comBlogger24125tag:blogger.com,1999:blog-5277161037604260037.post-85826618428851913782017-10-22T10:41:00.000-07:002017-10-22T10:41:03.954-07:00More than a shiny Prompt<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioNLElUuOaMWhFYtc__CQlrKl1IxRSa4DCftkFzB-8jTvsQKZDxdosxW-6xQ0qbxG7SxD9PYjXCnY_axPTmDwstQO967EO4GIAnaZTjqEWDHVtASXirADoMyTy0uE7ThsqpU3VLVIq9U8/s1600/shiny_dollar_sign.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" data-original-height="898" data-original-width="580" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioNLElUuOaMWhFYtc__CQlrKl1IxRSa4DCftkFzB-8jTvsQKZDxdosxW-6xQ0qbxG7SxD9PYjXCnY_axPTmDwstQO967EO4GIAnaZTjqEWDHVtASXirADoMyTy0uE7ThsqpU3VLVIq9U8/s200/shiny_dollar_sign.jpg" width="128" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Shiny Prompt</td></tr>
</tbody></table>
Recently, I have been assisting expect-lite script writers with their scripts, and the underlying problem was prompts. Prompt detection is very important in automating interactive programs. Fortunately, expect-lite allows for custom prompts (called user-defined prompts), but they can sometimes be tricky to define.<br />
<br />
<h4>
How do prompts work?</h4>
We use prompts without thinking what they really mean. A prompt is a way for the computer (or shell) to is printed to the screen when the current command is finished, usually a $, %, >, or #. It is the heart of an interactive program which is asking for user input.<br />
<br />
With expect-lite, the prompt indicates that the computer is ready to receive the next command. Much work has gone into expect-lite to detect the standard shell prompts. But what if you are automating a install script where the prompt is something like<span style="font-family: Courier New, Courier, monospace;"> '(yes/no)? '*</span><span style="font-family: inherit;">. How does one automate that?</span><br />
<span style="font-family: inherit;"><br /></span>
<h4>
<span style="font-family: inherit;">Two solutions</span></h4>
<br />
<ol>
<li>Use a user-defined prompt with the <span style="font-family: Courier New, Courier, monospace;">*/myprompt / </span>syntax. It is defined using regex and any characters in your prompt which are special to regex must be escaped with a backslash. Of the above yes/no prompt, the code would look like:<br /><span style="font-family: Courier New, Courier, monospace;">*/\(yes/no\)\? /<br />>yes</span><span style="font-family: inherit;"> </span></li>
<li><span style="font-family: inherit;">Use a non-regex expect line, followed by the answer with a no-wait-for-prompt send</span><br /><span style="font-family: Courier New, Courier, monospace;"><<yes/no<br />>>yes</span></li>
</ol>
<div>
While using the user-defined prompt is a more elegant solution, however it can be difficult to define the prompt without having a good knowledge of regex. The nice aspect of the second solution is although a bit wordy, no knowledge of regex is required. One only remember that the double >> means "do not wait for prompt".</div>
<br />
<span style="font-family: inherit;"><br /></span>
<h4>
<span style="font-family: inherit;">Interaction of expect lines << and prompts</span></h4>
<span style="font-family: inherit;">expect-lite is always searching forward when looking for an expect line. Take the following example:</span><br />
<span style="font-family: Courier New, Courier, monospace;">$ ssh 6palolo</span><br />
<span style="font-family: Courier New, Courier, monospace;">The authenticity of host '6palolo (2001:470:ebbd:0:221:2aff:fec3:6ab0)' can't be established.</span><br />
<span style="font-family: Courier New, Courier, monospace;">RSA key fingerprint is SHA256:tFStOlPO2dKJyqGE6D+7C8b1xu/4.</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;">Are you sure you want to continue connecting (yes/no)?</span><br />
<span style="font-family: Courier New, Courier, monospace;">yes </span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">And the following script:</span><br />
<span style="font-family: Courier New, Courier, monospace;">*/\(yes/no\)\? /</span><br />
<span style="font-family: Courier New, Courier, monospace;">>ssh 6pallolo</span><br />
<span style="font-family: Courier New, Courier, monospace;"><<continue connecting (yes/no)</span><br />
<span style="font-family: Courier New, Courier, monospace;">>yes</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">This script will <b>always</b> <b>fail</b>. The expect-lite search buffer is NOT line oriented. It is just searching a blob of text including new-lines. The reason the script will fail is that the third line, an expect line, will consume the text '(yes/no) ', and when the script continues and looks for the user-defined prompt it will no longer be in the search buffer, and the script will fail.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">A better way to write the script is to not expect what the user-defined prompt will look for:</span><br />
<span style="font-family: Courier New, Courier, monospace;">*/\(yes/no\)\? /</span><br />
<span style="font-family: Courier New, Courier, monospace;">>ssh 6pallolo</span><br />
<span style="font-family: Courier New, Courier, monospace;"><continue connecting</span><br />
<span style="font-family: Courier New, Courier, monospace;">>yes</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Lastly, method 2 can be used which does not require a user-defined prompt. expect-lite will wait for the text </span><span style="font-family: Courier New, Courier, monospace;">continue connecting</span><span style="font-family: inherit;"> before it sends </span><span style="font-family: Courier New, Courier, monospace;">yes</span><span style="font-family: inherit;">.</span><br />
<span style="font-family: Courier New, Courier, monospace;">>ssh 6pallolo</span><br />
<span style="font-family: Courier New, Courier, monospace;"><<continue connecting</span><br />
<span style="font-family: Courier New, Courier, monospace;">>>yes</span><br />
<h4>
<span style="font-family: Courier New, Courier, monospace;"><br /></span><span style="font-family: inherit;">Creating </span>consistent<span style="font-family: inherit;"> reproducible scripts</span></h4>
<span style="font-family: inherit;">In the end, you want to write a readable script which runs in a reproducible manner. Many times using method 2 (without user-defined prompts) is the easier path. But expect-lite allows you to create your own prompt detection, making</span> your life easier, because expect-lite is about automation for for the rest of us.<br />
<span style="font-family: inherit;"><br /></span>
<span style="font-size: x-small;"><span style="font-family: inherit;">* ssh will ask about continuing a connection to a new host by prompting with </span>(yes/no)? </span><br />
<span style="font-size: x-small;">** Reminder that << means expect without regex interpretation, and >> means send without waiting for prompt</span>Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-31215584386604297162017-02-23T12:30:00.000-08:002017-02-23T19:14:05.959-08:00Mature Code, not a bad thing<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiExYyG46hqNGBuyloxVuNjt_9R0VXnPDhEoCbo_UzGLCMsBmFedmiLUl0J_XFqBMCs6f3xX8uYgCZJ_RbJq6ilnr-EPLARYm7pdysdg06rVHalbuzpVJu_bp1kwVfQS_7NTRLedhHVJZo/s1600/Rpm_logo.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="127" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiExYyG46hqNGBuyloxVuNjt_9R0VXnPDhEoCbo_UzGLCMsBmFedmiLUl0J_XFqBMCs6f3xX8uYgCZJ_RbJq6ilnr-EPLARYm7pdysdg06rVHalbuzpVJu_bp1kwVfQS_7NTRLedhHVJZo/s200/Rpm_logo.png" width="200" /></a></div>
There is a certain attractiveness to shiny and new things. That can apply to programming languages, which is why we see newer languages like ruby, lua, and swift. They have got to be good, after all they are new and shiny.<br />
<br />
But there is the other side of the coin, stable mature code which doesn't change or break year after year. I chalk this principle up to why we still see code in c, a venerable language which is just a couple years short of turning 50.<br />
<br />
<h4>
Stable Code</h4>
Certainly, expect-lite is no c, but it is now twelve (12) years old, and I have always paid attention to making it backward compatible from its early days. The idea is not to break your code. One can spend way too much time troubleshooting broken code, that turns out the underlying infrastructure is what is broken your code.<br />
<br />
<h4>
RPM no longer working</h4>
Unfortunately, not everyone pays close attention to backward compatibility. It was recently pointed out to me that the expect-lite rpm would not install on RHEL 7 machines. It installed fine on the previous version, but something changed.<br />
<br />
To be fair, I wasn't creating the RPM the Redhat way. I don't run a Redhat machine these days, having migrated to Debian/Ubuntu years ago. I was using a program called <b>alien</b> to create the RPM from a 'deb' package file. And something was no longer working.<br />
<br />
<h4>
A new RPM</h4>
Fortunately, one of the expect-lite users, volunteered to make the RPM on RHEL 7, and that has been published to the <a href="https://sourceforge.net/projects/expect-lite/files/expect-lite/expect-lite_4.9.0/">SourceForge</a> site. If you use RHEL or Centos, you should run over and grab the new-working RPM from SourceForge.<br />
<br />
Thanks to the expect-lite community for making expect-lite automation for the rest of us.<br />
<br />
<span style="font-size: x-small;">* rpm logo <a href="https://commons.wikimedia.org/wiki/File:Rpm_logo.png">creative commons</a></span>Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-1508276224673966642016-11-12T15:25:00.001-08:002016-11-12T15:25:25.683-08:00Automating small tasks<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIwwn39BnW9yr76SDh_JAjHho2BHbdEtVB9Ig2kUtUY7BXlITWSqb7hcshqXCa3cXZB_rLC_4K9vp2nbNUrCQFWrMMIwoeWfEMivmv-nGaIBnk3m3MQcdqZLk881JDEdsB5uejxKTq7Ms/s1600/1024px-Glazed-Donut.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="130" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIwwn39BnW9yr76SDh_JAjHho2BHbdEtVB9Ig2kUtUY7BXlITWSqb7hcshqXCa3cXZB_rLC_4K9vp2nbNUrCQFWrMMIwoeWfEMivmv-nGaIBnk3m3MQcdqZLk881JDEdsB5uejxKTq7Ms/s200/1024px-Glazed-Donut.jpg" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Doughnut Script</td></tr>
</tbody></table>
In life we do small tasks over and over, thinking, this is too small to automate. After all, it may take 10 minutes to automate it, and it only takes 10 seconds to do it.<br />
<br />
Recently, I have been doing some testing on systemd, software for Linux. And I have been logging into the a DUT (Device Under Test) on a console cable. Using the console allows one to bring up and down the network interfaces, and still remain connected to the DUT.<br />
<br />
It didn't take long to log in via the console, start screen, log into the DUT, then su to root, about 10 seconds total. Then, type the commands to run the test, log out, and quit screen.<br />
<br />
I have been doing this for a couple of months, when I thought, hey, maybe I should sit down and automate this. Not because I was tired of logging in manually, but because I wanted to write a larger regression script for bugs I had raised against systemd.<br />
<br />
<h4>
Creating a doughnut script</h4>
So I spent 10 minutes writing and debugging a console login doughnut script*. Then I copied the script, and started adding my regression lines (in the middle). Since this is systemd, the regression test doesn't ever seem to pass, but at least I can quickly determine what is broken.<br />
<br />
Because I am never sure the state of the DUT, the serial login script does prompt detection to determine the next step in logging in. It does this with:<br />
<span style="font-family: "courier new" , "courier" , monospace;"># set user prompt</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">*/.*: /</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"># detect prompt</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">>>^M</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">+$prompt1=\n.*(\$ |login:|word:)</span><br />
<div>
<br /></div>
It determines prompts they same way you would with your eyes, presses <return> and "sees" what is returned. If it returns a prompt ($), then we are already logged in. If it asks for a login, then the script logs in.<br />
<br />
<h4>
Code Reuse</h4>
By setting the actual serial login in an include file, it is easily added to any script going forward. Suddenly the 10 minute time investment is paying off. But save yourself 10 minutes, and get the scripts from my <a href="https://github.com/cvmiller/expect-lite/tree/master/examples">github examples directory</a> (look for serial_login.elt, serial_con.inc, and serial_discon.inc).<br />
<br />
I find myself using the serial console script all the time, and it takes less than a second to log in. Automating little repetitive tasks can make life easier and faster, truly automation for the rest of us.<br />
<br />
<span style="font-size: x-small;">* a doughnut script is a script that sets things up, pauses (with *INTERACT), then breaks things down and cleans up.</span><br />
<span style="font-size: x-small;">** doughnut image by <a href="https://commons.wikimedia.org/wiki/User:Evan-Amos" title="User:Evan-Amos">Evan-Amos</a> - <span class="int-own-work" lang="en">Own work</span>, Public Domain, <a href="https://commons.wikimedia.org/w/index.php?curid=11918088">Link</a></span><br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-15897702898510223582016-01-07T10:02:00.000-08:002016-01-07T10:02:59.838-08:00Demystifying Regex Redux with 9 simple terms<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXqXj5B0edcOBmFDk4wPHpL8FPS-A0G7G_sAe97t-tR5n28THPOzIzbKfroS2b2UPUliU8yFkWXOOTwaBxafWUewmrn7m4axqLqgyCZRzKwWTty6g4WF9gzvBT8tlD2oze6WVOMG4PDI4/s1600/voting_regex.png" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXqXj5B0edcOBmFDk4wPHpL8FPS-A0G7G_sAe97t-tR5n28THPOzIzbKfroS2b2UPUliU8yFkWXOOTwaBxafWUewmrn7m4axqLqgyCZRzKwWTty6g4WF9gzvBT8tlD2oze6WVOMG4PDI4/s200/voting_regex.png" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Can I have your number?</td></tr>
</tbody></table>
When I wrote <a href="http://expect-lite.blogspot.com/2012/09/demystify-regex-with-7-simple-terms.html">demystifying regex with 7 simple terms</a> a while ago, I left out a couple really useful regex terms. So I guess this would have to be re-written as 9 Simple Terms.<br />
<br />
Regex is one of those things that when you need it, you need it. But it is from the 80's and cryptic. Most regex expressions you see are too complex, and hard to follow. In this post, I'll show you a couple more terms to help you keep it simple and to a minimum, while allowing you to tap the power of regex.<br />
<br />
expect-lite has very good support for regex meta characters (the ones that start with a backslash "\"). As a quick review of the 7 terms, there are:<br />
<ul>
<li>Repeats: * and +</li>
<li>Meta characters: \d, \w, \n, \t</li>
<li>Or: |</li>
</ul>
But there are a couple of regex meta characters which I have found useful in addition to the 7 above, when skipping over some columnar info to get that column you want to validate (or capture into a variable).<br />
\s is whitespace (space, tab, or newline)<br />
\S is <b>not</b> whitespace<br />
<br />
Working with the example from <a href="http://expect-lite.blogspot.com/2012/09/demystify-regex-with-7-simple-terms.html">demystifying regex with 7 simple terms</a>:<br />
<span style="background-color: white; color: #222222; font-family: "courier new" , "courier" , monospace; line-height: 14.784px;"><span style="font-size: x-small;">$ route -n<br />Kernel IP routing table<br />Destination Gateway Genmask Flags Metric Ref Use Iface<br />10.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0<br />169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 eth0<br />0.0.0.0 10.1.1.1 0.0.0.0 UG 100 0 0 eth0</span></span><br />
<br />
<span style="background-color: white; color: #222222; line-height: 14.784px;"><span style="font-family: inherit;">You could match the first IP address:</span></span><br />
<span style="font-size: x-small;"><span style="background-color: white; color: #222222; font-family: "arial" , "tahoma" , "helvetica" , "freesans" , sans-serif;"><span style="font-family: "courier new" , "courier" , monospace;">>route -n</span></span><br style="background-color: white; color: #222222; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; line-height: 14.784px;" /><span style="background-color: white; color: #222222; font-family: "arial" , "tahoma" , "helvetica" , "freesans" , sans-serif;"><span style="font-family: "courier new" , "courier" , monospace;"><\d+.\d+.\d+.\d+ </span></span></span><br />
<br />
But what if you wanted to match the default route metric (on the last line) , rather than the first IP address, you could use (I have highlighted the value we are searching for in bold):<br />
<span style="background-color: white; color: #222222; font-family: "arial" , "tahoma" , "helvetica" , "freesans" , sans-serif; font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;">#validate metric</span></span><br />
<span style="background-color: white; color: #222222; font-family: "arial" , "tahoma" , "helvetica" , "freesans" , sans-serif; font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;">>route -n</span></span><br />
<span style="background-color: white; color: #222222; font-family: "arial" , "tahoma" , "helvetica" , "freesans" , sans-serif; font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;"><0.0.0.0\s+</span></span><span style="background-color: white; color: #222222; font-family: "courier new" , "courier" , monospace; font-size: x-small;">\d+.\d+.\d+.\d+\s+0.0.0.0</span><span style="background-color: white; color: #222222; font-family: "courier new" , "courier" , monospace; font-size: x-small;">\s+UG\s+<b>100</b></span><span style="background-color: white; color: #222222; font-family: "courier new" , "courier" , monospace; font-size: x-small;">\s+\d\s+\d\s+eth0</span><br />
<br />
This introduces a new meta character, \s which is a space (or any white space). But it starts to look complex. Keeping it simple, use the non-space \S, and back tracking from a known point (in this example 'eth0') will simplify the regex some:<br />
<span style="background-color: white; color: #222222; font-family: "arial" , "tahoma" , "helvetica" , "freesans" , sans-serif; font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;">#validate metric</span></span><br />
<span style="background-color: white; color: #222222; font-family: "arial" , "tahoma" , "helvetica" , "freesans" , sans-serif; font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;">>route -n</span></span><br />
<span style="background-color: white; color: #222222; font-family: "arial" , "tahoma" , "helvetica" , "freesans" , sans-serif; font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;"><<b>10</b></span></span><span style="background-color: white; color: #222222; font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>0</b>\s+\S+\s+\S+\s+eth0</span><br />
<br />
It is better, but could be simpler by keying off of the flags column.<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">#validate metric of default route</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">>route -n</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><UG\s+<b>100</b>.+eth0</span><br />
<div>
<br /></div>
This uses the space meta-character, \s, and uses another meta-character mentioned in the fine print of the original post, the dot, or '.' which matches any character. It is good to use the dot sparingly, as it can often match more than you would expect. But in this example, because there is only one default route (it is after all, only IPv4*), and it is always on the last line, it is pretty safe to use the dot.<br />
<br />
To see just what expect-lite did match, use the *EXP_INFO directive on the CLI or earlier in your script.<br />
<br />
<h4>
Regex Guidelines</h4>
It is a good idea to keep regex as simple as possible. As we saw above, it is easy to create complex regex, but that leads to challenges in maintaining code later. Every time someone has to debug the script, they have to figure out what the regex is doing. Shorter, simpler regex will always win out.<br />
<br />
Regex has the concept of anchors (^,$), but I haven't included them the 9 simple terms because of a couple of reasons:<br />
<br />
<ul>
<li>Anchors don't work as you would expect in expect-lite. One would expect that you could use an anchor at the beginning of a line, but expect-lite doesn't evaluate output on a line by line basis, but rather a blob of text which includes new-lines. Therefore, if you need to "anchor" your regex, do something like '\n169.254.0.0' Regexs with anchors tend to not be simple or short<br /></li>
<li>I have seen regexs where the entire line is described, from beginning of the line to the end of the line, with anchors at each end. This almost always makes a very complex and brittle regex. A change in the column width, can break these kinds of regexes. Rather, it is much easier, and less code intensive to do a sparse validation of output using simple regexes (as shown in the example above).</li>
</ul>
<br />
<br />
Not everyone is a regex expert. Plan on helping the next person who looks at your script by writing a comment about what the regex is doing. And if you are lucky enough to be the next person to look at your code, then you will be thankful that you wrote your future-self a note.<br />
<br />
<h4>
Recap the 9 terms</h4>
To recap, and give you a single place to look for a reference, the 9 terms are the single character meta-characters:<br />
<br />
<ul>
<li>\d is a number</li>
<li>\w is a letter</li>
<li>\n is a new line (think of it as a carriage return)</li>
<li>\t is a tab</li>
<li>\s is white space (including \t and \n)</li>
<li>\S is a non-space (any letter, number, symbol)</li>
<li>. is any character (use this sparingly**)</li>
</ul>
<br />
<br />
And the repeat characters which are modifiers to the terms above:<br />
<br />
<ul>
<li>* repeats 0 or more times</li>
<li>+ repeats 1 or more times</li>
</ul>
<br />
<br />
And the regex OR term, |<br />
<br />
<h4>
The power of 9</h4>
You can still use only the original 7 regex terms and accomplish 90% of what you need. The additional 2 meta-characters just give you a bit more control over matching. And for those of us with a finite memory, it is still fewer than the fingers on two hands.<br />
<br />
<br />
<span style="font-size: x-small;">* IPv6 can often have multiple default routes, and the metric becomes very important in determining which one is used.</span><br />
<span style="font-size: x-small;">** the regex dot is extra credit</span><br />
<br />
<br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-20648365006381929692015-09-15T10:46:00.001-07:002015-09-15T10:46:59.058-07:00Sleeping while you workSleeping: restful, relaxing, restorative. But sleeping in a computer script is pausing the script for a specified amount of time. As of <b>version 4.9.0 </b>a native sleep command, using a colon, has been added.<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2VT8dbCCQ25Jqk4KadTesqwrAoilZMVhE6paC3nxXGP-Vh7sP-GIsi_jhpkrpkAKsfG86UFvwWifsaPpOeV8n-c5X5x7HoWI714O8sekHSyQuNw9I43nMASXZhCgq-4uxbbNMXcGtKso/s1600/westport_library_clock_tower_workings%252B.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2VT8dbCCQ25Jqk4KadTesqwrAoilZMVhE6paC3nxXGP-Vh7sP-GIsi_jhpkrpkAKsfG86UFvwWifsaPpOeV8n-c5X5x7HoWI714O8sekHSyQuNw9I43nMASXZhCgq-4uxbbNMXcGtKso/s320/westport_library_clock_tower_workings%252B.jpg" width="288" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Counting the seconds</td></tr>
</tbody></table>
<br />
"Why wait until expect-lite has been around for 10 years before adding sleep?" you may ask. Because I have seen sleep abused in other scripting languages, usually a scripter will add a 60 to 600 second sleep rather than check for the event with a polling loop.<br />
<br />
<h4>
Polling with a sleep</h4>
But polling loops are an excellent example of where to use a sleep. It is usually unnecessary to check for a state change (ethernet interface up, for example) on a mili-second time basis. More likely if the interface comes up in a couple of seconds, that is good enough. The sleep will slow down the polling loop so the loop does not put an undo load on the machine.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># using a polling loop to check when eth0 is up</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$intf=eth0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$int_state=none</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[ $int_state != UP</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ip link show dev $intf</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> +$int_state=(UP|DOWN)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> # sleep 2 seconds to slow down loop</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> :2</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">]</span><br />
<br />
<h4>
Wait a sec :01</h4>
The colon ":" indicates a sleep. In the above example :2 is used to sleep (or pause) the script for 2 seconds. Sleep is always in seconds, but mili-seconds are also supported such as 5 mili-seconds:<br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> :0.005</span><br />
<br />
The native sleep (using the colon) also gives indication that the script is sleeping. There is nothing more frustrating that debugging a script and wondering is it hung or is it sleeping. expect-lite will print <i>dots</i> to show the progress of the sleep. A 12 second sleep would output:<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Sleeping: 12 </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">....+....10.. </span><br />
<div>
<br /></div>
Each dot represents a second, with the plus every 5 seconds, and a number every 10 seconds. This output will go to stdout, and also be logged to a file with the *LOG command so that it can be observed later. The output can be disabled with the *NOINFO command if you want less clutter.<br />
<br />
<h4>
Transparency in sleeping, making scripting easier</h4>
A goal of expect-lite is transparency, showing you what it is doing, to help you debug script errors, or determine actual problems with the device you are testing. And now, you can sleep on it.<br />
<br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-42330210088645105982015-01-11T17:01:00.000-08:002015-01-11T17:01:09.372-08:00Looking through the window, the expect-lite debugger<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZWI8GmE2aVSQS637XDzkZEGgZvF64RGzyf4nJTHy1PhG1a1ebCU8aESEfvmlNNk4doGEUOnGUNmi7jD2xIMD_WrOY-5Yy1bqS0x27c6dZ91ZelS5OnSm7DHGRVgH7O-qLRAc3ReVdbD4/s1600/gatienau2_window_snow_view+.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZWI8GmE2aVSQS637XDzkZEGgZvF64RGzyf4nJTHy1PhG1a1ebCU8aESEfvmlNNk4doGEUOnGUNmi7jD2xIMD_WrOY-5Yy1bqS0x27c6dZ91ZelS5OnSm7DHGRVgH7O-qLRAc3ReVdbD4/s1600/gatienau2_window_snow_view+.jpg" height="200" width="156" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Looking through the debugger</td></tr>
</tbody></table>
In January 2015, expect-lite will have been making automation easier for 10 years. In celebration of that event, version 4.8.0, includes improvements to the debugger to make automation even easier.<br />
<br />
expect-lite has had an integrated debugger, called with *INTERACT, since version 4.0 (Oct 2010).The expect-lite debugger allows you to:<br />
<ul>
<li>all the standard debugger stuff: step, skip, view variables</li>
<li>type commands directly to the device/host the script is connected to</li>
<li>execute arbitrary lines of expect-lite script (copy/paste uses this as well)</li>
</ul>
The debugger has the standard things you would look for in a debugger. Pressing <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">escape h</span> will print the debugger help:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">IDE: Help</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> Key Action</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ---- ------</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <esc>s Step</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <esc>k sKip next step</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <esc>c Continue</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <esc>v show Vars</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <esc>e show Env</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <esc>0to9 Show N next lines of script</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <esc>-1to-9 Show N previous lines of script</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ctrl+D Quit & Exit expect-lite</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <esc>h this Help</span><br />
<br />
The debugger is like a window to the device being automated. Once in a breakpoint, the debugger silently steps aside, and allows you to type directly to the device. Perhaps you aren't getting the response you expected, or the device wasn't configured as you had expected. You can fix it while in the middle of your script.<br />
<br />
The debugger silently watches what you type, and decides if the text is for the device, or is it an expect-lite command it must execute. How does it do this? Mostly pretty well.<br />
<br />
<b>Typing expect-lite commands in the debugger</b><br />
<br />
You may have noticed that expect-lite commands start with a punctuation character like '>', '<', '?', '@', or ';'. The debugger watches for these characters at the beginning of a line. But there are some tweaks to prevent the debugger from accidentally grabbing a line that was intended for the device. These are limitations only when typing in the debugger, and are not required when executing the expect-lite script.<br />
<ul>
<li>>send this There must be <i>no</i> space between > and the text to be sent</li>
<li>; comment There <i>must</i> be a space between the semi-colon and the comment</li>
<li>?if condition... When using an IF statement in the debugger then the optional 'if' must be used, there are just too many question marks in normal text</li>
<li>$var=value Variable assignments <i>must</i> have an equals sign (no spaces)</li>
<li><i>no</i> white space before the expect-lite command (punctuation)</li>
</ul>
I wrote in an earlier blog entry (<a href="http://expect-lite.blogspot.ca/2012/12/writing-scripts-with-copy-and-paste.html">writing scripts with copy and paste</a>), that using the debugger you can also copy/paste into a running script. When copy/pasting into the debugger, only the first line need follow the above limitations, the remaining lines can have leading white space, etc, since the debugger has decided that the entire paste is expect-lite script.<br />
<br />
<b>*SHOW ENV</b><br />
<br />
The biggest improvement to the debugger in version 4.8.0, is *SHOW ENV. Similar to *SHOW VARS, the expect-lite environment consists of a list of directives which are enabled/disabled, such as *INFO, and *TIMESTAMP, as well as ones with values like user defined prompt, and the infinite loop count (*INFINITELOOP). Lastly, *SHOW ENV will display any shell environment variables which begin with EL_.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ DEBUG Info: Printing expect-lite directives/environment</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Environment: Value:</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">CURRENT_FORK_SESSION default</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">DEBUG off</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">DVPROMPT on</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">EOLS LF</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">EXP_INFO on</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">FAIL_SCRIPT fail_test.inc</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">INFINTE_LOOP_COUNT 5000</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">INFO on</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">LOG off</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">LOG_EXT .log</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">NOFAIL 2</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">NOINCLUDE off</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">NOINTERACT off</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">REMOTE_SHELL none</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">TIMESTAMP off</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">USER_DEFINED_PROMPT .*$ $</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">WARN on</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">fuzzy_range 10</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">timeout 2</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">EL_CONNECT_METHOD ssh_key</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">EL_REMOTE_HOST localhost</span><br />
<div>
<br /></div>
Of course, if you are in the debugger, you don't have to type *SHOW ENV, you can just type <esc>e.<br />
<br />
Just as you can start a new shell, by typing 'bash' or 'csh' at the prompt. There is an included example called 'el_shell.elt' which you can use to create an expect-lite shell. It is a simple script which just drops you into the debugger. From there, you can type regular linux commands (or cygwin linux-like commands), expect-lite commands, or explore the debugger help with <esc>h.<br />
<br />
<b>Doughnut Scripts</b><br />
<br />
I find I often create a doughnut script, a script with a hole in the middle. I use this to have a script set up my environment, then drop to the debugger, allowing me to plunk around, then when I am done, I exit the debugger (with '+++'), and the script cleans up. I call this automation assist. It doesn't do everything, but it allows me to explore/test faster than doing it all by hand.<br />
<br />
The debugger command detection isn't perfect, but it works pretty well. Sometimes you may find a smudge or a dirty spot on the debugger window, but hopefully, it will be clear enough to see that expect-lite is automation for the rest of us.<br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-68992072094825112302014-12-22T18:07:00.002-08:002014-12-22T18:07:45.659-08:00A tale of two scripts redux: joining Python and expect-lite<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEWiqGlzFHpuW9kQgfhNJw0BjKHBFLyP-sJ4XFpIuY9ydZPpgmOLr5TWO_4oThzjrR-Gf-PhKSe7d7xk3vwoKjfL-HmK_AvyToDpgkrD3sQocaNIaMXefh8n8-125pVc0ppRxyCCziWro/s1600/snake_moulting_cls+.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEWiqGlzFHpuW9kQgfhNJw0BjKHBFLyP-sJ4XFpIuY9ydZPpgmOLr5TWO_4oThzjrR-Gf-PhKSe7d7xk3vwoKjfL-HmK_AvyToDpgkrD3sQocaNIaMXefh8n8-125pVc0ppRxyCCziWro/s1600/snake_moulting_cls+.jpg" height="200" width="175" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Python wrapper</td></tr>
</tbody></table>
As I mentioned earlier, expect-lite interpreter is quite basic. The advantage of such simpleness is that it ignores anything it doesn't understand. In<a href="http://expect-lite.blogspot.ca/2012/08/a-tale-of-two-scripts.html" target="_blank"> a tale of two scripts</a> I explained how to join bash and expect-lite in the same script. But the technique can also be applied to other languages.<br />
<br />
In this post, I'll show how expect-lite can be embedded in a Python script allowing you to get all the goodness of Python <i>and</i> expect-lite.<br />
<br />
First we start with a simple Python script which just makes a call to subprocess (used to execute an external application), prints the output of the expect-lite script in real time and then checks the return code of the subprocess.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">#!/usr/bin/env python</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">"""</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Example script: embedding expect-lite in python</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> 22 December 2014 -- Craig Miller</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">"""</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import subprocess, inspect, os</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">def embed():</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> script_name = inspect.getfile(inspect.currentframe())</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> process = subprocess.Popen('expect-lite ' + script_name, </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> shell=True, stdout=subprocess.PIPE)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> # print stdout in real time</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> line = None</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> while line != '':</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> line = process.stdout.readline()</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> print "EL:", line.rstrip()</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> out, err = process.communicate()</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> if process.returncode > 0:</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> print("========= ERROR:%s" % process.returncode)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> else:</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> print("========= GOOD:%s" % process.returncode)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">#############################</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># beginning of python script</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">if __name__ == '__main__':</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> try:</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> embed()</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> except KeyboardInterrupt:</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> print "Detected ^C"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> os._exit(1)</span><br />
<div>
<br /></div>
<div>
If you are familiar with Python, then you know that indentation is more than just a a good idea, it is <i>required</i> by the language. The script above has the basic components of a Python script, but the important part is the function <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">embed()</span>. </div>
<div>
<br /></div>
<div>
In <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">embed()</span>, it figures out the name of the script using <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">inspect.getfile() </span><span style="font-family: inherit;">and saves it in the </span>variable<span style="font-family: inherit;"> </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">script_name</span><span style="font-family: inherit;">. Then </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">subprocess.Popen() </span><span style="font-family: inherit;">is called with expect-lite and the </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">script_name</span><span style="font-family: inherit;">, to recursively run the script again, this time using expect-lite as the interpreter. </span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">However, before we do that, we need to add the expect-lite part of the script. At the bottom of the Python script add:</span></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">if False:'''</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">#############################</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># beginning of expect-lite code</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">*EXP_INFO</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$count=3</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">; === test of EL</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">>echo "inside expect-lite! whoo-hoo"</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><inside</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">@5</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">;purple === ping loopback</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">>ping6 -c $count ::1</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><packets transmitted</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">>echo "Continue"</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">; === pau</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">'''</span></div>
</div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">The trick is to <i>protect</i> the expect-lite lines from the Python interpreter. The Python interpreter is a bit smarter than the bash interpreter, which does not read the entire file before executing. So to make python happy, we must shield the expect-lite lines inside an </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">if false</span><span style="font-family: inherit;"> statement. The triple quotes is a Python mechanism that allows anything to be entered, even expect-lite. Save the completed script as two_scripts.py</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">The expect-lite part of the script is a simple one, showing that we are actually inside the expect-lite script, and a simple ping6 of the IPv6 loopback address.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">Because of the dual nature of the script, it is possible to run (this example) without any Python, by running the </span>script<span style="font-family: inherit;"> directly from expect-lite.</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">expect-lite two_scripts.py</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">Or run it using Python:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">python two_scripts.py</span></div>
<div>
<br /></div>
<div>
The expect-lite part of the script can be located anywhere in the python script, as long as <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">if false</span> is used to protect it. Of course, you can also have Python pass parameters to expect-lite using <a href="http://expect-lite.blogspot.ca/2012/07/the-power-of-constants.html" target="_blank">CLI constants</a>, making the script even more flexible and useful. </div>
<div>
<br /></div>
<div>
There you have it, two languages, one script, all the power of Python, all the simplicity of expect-lite, automation for the rest of us.<br />
<br />
Happy Holidays!</div>
<div>
<br /></div>
Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-27819777088986314602014-11-19T17:21:00.001-08:002014-11-19T17:21:49.895-08:00Sawing Logs<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihqwxED3qOcTdHuk8Z7zPS1H4dldn9P5S8ZiuLcEx65Nmikf-2qzl73pybb4PLbxFoG4hk-dBnG8eKt1IiJr8UWJkxf9wGh0ddV3PnbKdFxQMXDt2xTwhNC0WxZN7-y1dzLxtaforT74I/s1600/tree_cut_chop_log2+.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihqwxED3qOcTdHuk8Z7zPS1H4dldn9P5S8ZiuLcEx65Nmikf-2qzl73pybb4PLbxFoG4hk-dBnG8eKt1IiJr8UWJkxf9wGh0ddV3PnbKdFxQMXDt2xTwhNC0WxZN7-y1dzLxtaforT74I/s1600/tree_cut_chop_log2+.jpg" height="187" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sawing Logs the way you like them</td></tr>
</tbody></table>
Recording what happened for later review is called logging. For years expect-lite lacked native logging, and relied on other programs like 'script' or 'tee' to record the output. Logging was added to make it easier to record just the parts you might want to keep for later.<br />
<br />
expect-lite uses directives to control behavior of operation. Directives always start with an asterisk, and all CAPS, like *INFO. The *LOG, *NOLOG and *LOGAPPEND directives added native logging support in 2013. Like all directives, the log commands can be used on the command line when starting a script, or within the script to just log the desired portion.<br />
<br />
<h4>
*LOG</h4>
The *LOG will automatically create a log file with the <script_name>.log in the script directory. But *LOG can also take a path/filename parameter to log to a different directory.<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$path=/tmp</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">; === get today's date</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">>date %F</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+$today=\n(\d+-\d+-\d+)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">*LOG $path/$arg0-$today.log</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">>do stuff</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<br />
<h4>
*NOLOG</h4>
And the *NOLOG stops the logging. For example, perhaps there is a while loop that is polling for some event, you don't really need to see that it polled 500 times in the log, you just want to see (in the log) what happens after the event.<br />
...<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">*LOG $path/myfile-$today.log</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">>do stuff</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">*NOLOG</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[ $state != $event</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ; poll for event</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> >show event</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> +$state=(enabled|disabled)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">]</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">*LOGAPPEND </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">$path/$arg0-$today.log</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">...</span><br />
<br />
And *LOGAPPEND will append to an exiting log file, or create a new file, if there is no existing file.<br />
<br />
<h4>
What is logged</h4>
The *LOG log file will capture everything that is seen on the terminal screen, expect-lite messages such as *INFO, *WARN, commands typed in during an *INTERACT (a breakpoint), and all standard out, including colour. (hint use *NOCOLOUR to disable colour output).<br />
<br />
<h4>
Plays well with Instant Interact</h4>
Why not just use 'script' or 'tee' commands to record the script output. No reason, if that it working for you. However *LOG provides better control over what goes into the log file. Additionally, I had real problems using 'tee' and Instant Interact (creating a breakpoint on the fly) with Ctrl+\, as it would terminate the 'tee' command (and in turn the running script). Using native logging with *LOG allows you to use Ctrl+\ (control backslash) at any point during the script, giving you immediate access to the running script without having to preset a breakpoint.<br />
<br />
So go ahead and saw those logs up anyway you like, because expect-lite is automation for the rest of us.<br />
<br />
<span style="font-size: x-small;">Note 1: the pre-defined variable $arg0 holds the script name</span><br />
<span style="font-size: x-small;">Note 2: for those who don't require 'U's in colour, *NOCOLOR will also work</span><br />
<span style="font-size: x-small;"><br /></span>Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-13122317305987202152014-10-19T19:54:00.000-07:002014-10-19T19:54:30.497-07:00Capturing the State of Mind<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi18kDkUybVR91HuLuvGfFNAULUoa-TLbFBdH8ds4rhZP9VrCjqPCIdUMdzqs6kD-Bbqf_y3IOs_j9cas9asSBsifRExsYoTXvlIyuze_l45V3iKfxOetPiB4d4zqcL2NwhCinQaCvt6tM/s1600/fourth_line_buddah+.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi18kDkUybVR91HuLuvGfFNAULUoa-TLbFBdH8ds4rhZP9VrCjqPCIdUMdzqs6kD-Bbqf_y3IOs_j9cas9asSBsifRExsYoTXvlIyuze_l45V3iKfxOetPiB4d4zqcL2NwhCinQaCvt6tM/s1600/fourth_line_buddah+.jpg" height="179" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">State of Mind</td></tr>
</tbody></table>
There are things out there which have state. Think of your gas tank, could be full or empty. Maybe it is an interface that is up or down, or initializing. You may need to know the state before your expect-lite script can continue.<br /><br />In this blog entry I'll cover a key tip of grabbing just the state of a device by using one of the <a href="http://expect-lite.blogspot.ca/2012/09/demystify-regex-with-7-simple-terms.html" target="_blank">simple 7 terms</a> from regex, the OR.<br /><br />For example, perhaps you want to test an ethernet interface, but it would be really useful if the interface such as wlan0 was UP before starting your test.<br /><br /><span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">$ ip link<br />1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN mode DEFAULT <br /> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00<br />3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000<br /> link/ether 00:16:cb:b4:7c:52 brd ff:ff:ff:ff:ff:ff</span></span><br /><br /><br />Or the ifconfig command, however it doesn't show the interface as DOWN.<br /><br />$ /sbin/ifconfig -a<br /><span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"><br />wlan0 Link encap:Ethernet HWaddr 00:16:cb:b4:7c:52 <br /> BROADCAST MULTICAST MTU:1500 Metric:1<br /> RX packets:0 errors:0 dropped:0 overruns:0 frame:0<br /> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br /> collisions:0 txqueuelen:1000 </span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)</span></span><br /><br /><br />So using a dynamic variable we can capture the state of the interface, letting expect-lite to do all the hard work of parsing through the output. Since the 'ip' command is less typing, and newer, I"ll use it in my examples.<br /><br /><span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">$interface</span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">>ip link<br /><$interface<br />+$interface_state=state (UP|DOWN)</span></span><br /><br />There are a few things occurring in this short script which are covered in older posts:<br /> 1. <span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">$interface</span></span> by assigning it to a variable, it is possible to use the <a href="http://expect-lite.blogspot.ca/2012/07/the-power-of-constants.html" target="_blank">power of constants</a> to override the variable on the command line, making the script more flexible.<br /> 2. <span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"><$interface</span></span> is an expect line, which <a href="http://expect-lite.blogspot.ca/2012/10/consuming-output-eating-tables-for-lunch.html" target="_blank">consumes the output</a> of the 'ip
link' command. Therefore making the dynamic variable capture on the next
line much easier. The first state encountered in the remaining output of the
'ip link' command is the one we want.<br /> 3. Lastly, and this is the <i>key</i> to this post, <span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">(UP|DOWN)</span></span> is using the regex OR '|' which means: capture either the word 'UP' or the word 'DOWN' and nothing else. The variable <span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">$interface_state</span></span> is guaranteed to only be one of those states.<br /><br />Now that it is easy to get the state of an interface how do we wait for the interface to change state? With a while loop, of course. Let's expand on the script we have above.<br /><br /><span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">$interface<br />>ip link<br /><$interface<br />+$interface_state=state (UP|DOWN)<br /># while $interface_state is not equal to UP<br />[ $interface_state != UP<br /> !sleep 2<br /> >ip link<br /> <$interface</span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> +$interface_state=state (UP|DOWN)<br />]<br />; === The $interface is now $interface_state</span></span><br /><br />Of course this could be improved, such as if the interface never comes 'UP', the script will spend a very long time in the while loop (eventually it will hit the infinite loop protection, and stop). Typically I add a counter variable, and check if the counter variable has exceeded a maximum number. But I'll leave that to you or another post.<br /><br />When capturing states in expect-lite, it is important to put all the states in capture parens e.g. (UP|DOWN|INIT|RESET) by doing this, you are guaranteed to always capture the state.<br /><br />Automating state is easy with expect-lite. State of mind... is a little harder.<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-78493375824526406542014-05-30T02:44:00.000-07:002014-05-30T02:44:30.097-07:00Saving several slices of output for later with Pseudo Arrays<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUo29BpKXUmGUSfir7NdnhskrNQhEvjik3dezELh1c1m9iTym4rFjV_ca0K3eBnTuIeminlmO5J62NKUpPvs488hdmg4f2LGtl0bz0tfC6SNXj7cnvulQVGlTHmhPQ5sz2fHAHTW_8Mps/s1600/denver_bixi_bike_parking+.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUo29BpKXUmGUSfir7NdnhskrNQhEvjik3dezELh1c1m9iTym4rFjV_ca0K3eBnTuIeminlmO5J62NKUpPvs488hdmg4f2LGtl0bz0tfC6SNXj7cnvulQVGlTHmhPQ5sz2fHAHTW_8Mps/s1600/denver_bixi_bike_parking+.jpg" height="138" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Pseudo Array of bikes</td></tr>
</tbody></table>
<div style="margin-bottom: 0in;">
<style type="text/css">P { margin-bottom: 0.08in; }</style>I was recently talking to a group of expect-lite users who wanted to know how to save several slices of the output of a command, and then be able to use those
slices later on. A simple while loop and a pseudo array would be a good solution to this problem.</div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
What is a pseudo array you ask? Well it
acts like an array, but it isn't an array in the strictest sense. A
real array would have the form $var(index), e.g. $myvar(5). </div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
But
expect-lite doesn't support real arrays, the format for pseudo array
is $var$index e.g. $myvar$i. In fact, when using pseudo arrays, new
variables names are automatically being created, e.g. $myvar1, $myvar2 ..
$myvar100, etc.</div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
An example problem is to collect the block devices
(like hard drives) in a pseudo array to be further examined later. In
this example, I'll use the <span style="font-family: Courier, monospace; font-size: x-small;">ls</span>
command to display the devices in /dev. The ones which are block
devices start with a 'b'. An example
output will look like:</div>
<div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">$
ls -l /dev</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">crw-------
1 root wheel 1, 0 May 4 07:14 auditpipe</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">crw-------
1 root wheel 13, 0 May 4 07:15 autofs</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">crw-------
1 root wheel 18, 0 May 4 07:15 autofs_control</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">crw-rw-rw-
1 root wheel 17, 3 May 4 07:15 autofs_nowait</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">crw-------
1 root wheel 23, 0 May 27 08:11 bpf0</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">crw-------
1 root wheel 23, 1 May 27 08:11 bpf1</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">brw-r-----
1 root operator 14, 0 May 4 07:14 disk0</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">brw-r-----
1 root operator 14, 2 May 4 07:14 disk0s1</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">brw-r-----
1 root operator 14, 1 May 4 07:14 disk0s2</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">brw-r-----
1 root operator 14, 3 May 4 07:14 disk0s3</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">brw-r-----
1 root operator 14, 4 May 4 07:14 disk0s4</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">brw-r-----
1 root operator 14, 5 May 4 07:14 disk0s5</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">brw-r-----
1 root operator 14, 6 May 4 07:14 disk0s6</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">brw-r-----
1 root operator 14, 7 May 4 07:14 disk0s7</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">crw-rw-rw-
1 root wheel 4, 0 May 4 07:14 ttyp0</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">crw-rw-rw-
1 root wheel 4, 1 May 4 07:14 ttyp1</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">crw-rw-rw-
1 root wheel 4, 2 May 4 07:14 ttyp2</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">crw-rw-rw-
1 root wheel 4, 3 May 4 07:14 ttyp3</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">brw-------
1 root operator 1, 0 May 4 07:14 vn0</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">brw-------
1 root operator 1, 1 May 4 07:14 vn1</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">brw-------
1 root operator 1, 2 May 4 07:14 vn2</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">brw-------
1 root operator 1, 3 May 4 07:14 vn3</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">...</span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<style type="text/css">P { margin-bottom: 0.08in; }</style>
<div style="margin-bottom: 0in;">
For this example, I'll use a while loop
to iterate through the output, and increment the pseudo array index
variable, capturing the block devices into a pseudo array of dynamic variables. If the
dynamic variable can not find a block device, then the variable will
be set to a special expect-lite value of __NO_STRING_CAPTURED__. The while loop will
continue looping until there are no more block devices.</div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
After
capturing the device in pseudo array variable <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$dev$i</span></span>, the script will consume the output (see
<a href="http://expect-lite.blogspot.ca/2012/10/consuming-output-eating-tables-for-lunch.html" target="_blank">consuming tables for lunch</a>) to remove the top part of the output. That will leave the next block device to be captured near the top of the blob of text output for the next iteration of
the while loop. I'll use some of the basic regex such as \n, \d, and \w (see <a href="http://expect-lite.blogspot.ca/2012/09/demystify-regex-with-7-simple-terms.html" target="_blank">demystify regex with 7 simple terms</a>) to ensure the line begins with 'b' and ends with the desired device name. Only the part in parens, (\w+), is <i>captured</i> into the dynamic variable (see <a href="http://expect-lite.sourceforge.net/expect-lite_doc.html#variables" target="_blank">expect-lite variables</a>).</div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">>ls
-l /dev</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">#
initialize index variable</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">$i=0</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"># initialize the first element in the pseudo array </span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">$dev$i=none</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">#
while loop testing the pseudo array value is captured</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">[
$dev$i != __NO_STRING_CAPTURED__</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> +$i</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> #
capture the block device</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> +$dev$i=\nb.*\d\d:\d\d
(\w+)</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> #
expect the device to consume the output</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> ?
$dev$i != __NO_STRING_CAPTURED__ ? <$dev$i</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">]</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"><br /></span></span>
</div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">#
set max devices captured</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">$max=$i</span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="font-family: Times New Roman, serif;"><span style="font-size: small;">Now
that the block devices have been captured into a pseudo array, I'll
explore them a bit more using the <span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">file</span></span> command. Using another while
loop to iterate through the pseudo array re-using the index.</span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">#
initialize index variable</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">$i=1</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"># while loop to check the devices in the pseudo array </span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">[
$i < $max</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> #
show the pseudo array value - the device</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> >file
$dev$i</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> #
check that it is a block special device</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> <block special</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> +$i</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">]</span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="font-family: Times New Roman, serif;"><span style="font-size: small;">Of
course you can always look at the pseudo array (and all the
expect-lite variables) in the IDE (debugger) by typing <esc>v
or you can print it out right from the script using the directive
<span style="font-family: Courier, monospace;"><span style="font-size: x-small;">*SHOW VARS</span></span>.
Depending on your system block devices, the output would look something like:</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">$
DEBUG Info: Printing all expect-lite variables</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">Var:arg0
Value:test_pseudo_array.txt</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">Var:dev0
Value:none</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">Var:dev1
Value:vn0</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">Var:dev2
Value:vn1</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">Var:dev3
Value:vn2</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">Var:dev4
Value:vn3</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">Var:dev5
Value:__NO_STRING_CAPTURED__</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">Var:i
Value:5</span></span></div>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">
</span></span><div style="margin-bottom: 0in;">
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">Var:max
Value:5</span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
So, not only can you eat your output for lunch, but you can save the slices in a pseudo array for a midnight snack. expect-lite, serving up automation for the rest of us.</div>
<div style="margin-bottom: 0in;">
<br />
</div>
Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-17876190175967747132014-04-20T07:09:00.001-07:002014-04-20T07:09:30.335-07:00Warm and Fuzzy<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvCDSIh0jp67kNzhDMEcMd9-cjEFox7XDDNHeeoYMYlecNg2aEeeO5NTX9daLbypEke9r2_ABL6anCp9PckEFgbG81tsYplrcLxUGAiNi-ebVpaWLo313IbNhyphenhyphen76TouVIeNn8pHB3LhI0/s1600/magnic_light_test_ride_fuzzy+.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvCDSIh0jp67kNzhDMEcMd9-cjEFox7XDDNHeeoYMYlecNg2aEeeO5NTX9daLbypEke9r2_ABL6anCp9PckEFgbG81tsYplrcLxUGAiNi-ebVpaWLo313IbNhyphenhyphen76TouVIeNn8pHB3LhI0/s1600/magnic_light_test_ride_fuzzy+.jpg" height="166" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Life can be Fuzzy</td></tr>
</tbody></table>
One particular problem I have been grappling with is how to do an approximate check of a value. Being a user, as well as the maintainer of expect-lite, I am interested in improving expect-lite with the goal to make things easier. <br /><br />
Say for example, you want to check the load on a machine, but if you have every looked at load, you know it isn't an exact number. The first load average number represents the average load on the
machine in the past minute (the other two are averages over 5 and 15
minutes).<br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-size: x-small;">$ uptime<br /> 17:10:17 up 98 days, 7:21, 12 users, load average: 1.39, 1.34, 1.03</span></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-size: x-small;"> </span></span><br />
Back to checking the load of the machine. With expect-lite you could capture the value into a dynamic variable, and then use an if statement to do a comparison, which is a very programatic approach.<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>uptime</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><load average</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">+$load=(\d\.\d+)</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"># fail if load is higher than 3.00 </span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">? $load > 3? *FAIL</span></span><br />
<br />
But if you had to do this for several values, such as all three load averages, it would grow tedious quickly. It certainly did for me.<br />
<br />
So I added the concept of a <b>fuzzy expect</b>, or expecting an approximate value (in version 4.7.0). With fuzzy expect, you must declare how much fuzziness you will permit. This is similar to setting the timeout value (e.g. @10) or setting a custom prompt (e.g. */< /), it remains in effect until you change it or the script ends. To set the fuzziness use, the following:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">~=1.5</span></span><br />
Then to apply this fuzziness to the load problem above, all that is required is:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>uptime</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><load average </span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">~<(1.5)</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"></span></span><br />
That's it! What the last line means is: expect the first number after 'load average' and compare it with the tolerance set with the earlier fuzziness value. If the 1 minute load average is N, then N must fall in the range of 0 to 3 (1.5 + or - the fuzzy value of 1.5).<br />
<br />
But wait, it could be simpler yet, since fuzzy expect can be combined with literal expect, like this:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>uptime</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">~<load average: (1.5) </span></span><br />
<br />The parens defines which part is fuzzy. Alas, expect-lite still has room for improvement, and it can only do one fuzzy expect per line. If you needed to check all three load averages, then you would have to add a couple more lines:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>uptime</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">~<load average: (1.5)</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">~<(1.5)</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">~<(1.5) </span></span><br />
<br />
That certainly is easier than sucking those three load average values into dynamic variables, and then running three separate if statements to do the check.<br />
<br />
Some additional qualities of the <b>fuzzy expect</b> are:<br />
<ul>
<li>Default fuzzy value is 10, change it to what you would like</li>
<li>Regex can be used (just like with '<')</li>
<li>Just like '<', matches consume the expect buffer (see <a href="http://expect-lite.blogspot.ca/2012/10/consuming-output-eating-tables-for-lunch.html" target="_blank">Consuming Output</a>)</li>
<li>For those who dream in hexidecimal, fuzzy numbers can be in hex (e.g. 0xdead)</li>
<li>Fuzziness only applies to numbers</li>
</ul>
Sometimes life is fuzzy (and warm), now expect-lite can help with that cashmere sweater(s).<br />
<br />
<span style="font-size: x-small;"><span>PS. you will remember from "<a href="http://expect-lite.blogspot.ca/2012/09/demystify-regex-with-7-simple-terms.html">Demystifying Regex</a>" that \d is a digit.</span></span><br />
<br />
<br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-77576850545939570712014-02-17T06:31:00.000-08:002014-04-20T07:12:28.447-07:00Including (beer) made easy<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAvLQ99OFd04JCwPbNwTKgtRoEnV7eL2TVSkNuJlxNe98MqxAoThsJZCyYY0zJiVxIFYNEPunVI-bq41el14dzNEBNXDPCx95BxF4kYaw1OE_O_-5Ej_ORprxs92jtXVkBlX5ytqZLjVE/s1600/2004_canada_day_beer_flag2_include.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAvLQ99OFd04JCwPbNwTKgtRoEnV7eL2TVSkNuJlxNe98MqxAoThsJZCyYY0zJiVxIFYNEPunVI-bq41el14dzNEBNXDPCx95BxF4kYaw1OE_O_-5Ej_ORprxs92jtXVkBlX5ytqZLjVE/s1600/2004_canada_day_beer_flag2_include.jpg" height="200" width="181" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Include Beer</td></tr>
</tbody></table>
Recently, I had a question from an expect-lite user about include files. Include files in their simplest form can be thought of an automated way to paste code into your script. And the original intent of an include file was just that. This concept is not a new in computer languages, c has header files, bash can source files, python imports files, etc. But as expect-lite has evolved, I'd like to say improved, and so have the uses of include files.<br />
<br />
There are two types of include files,<br />
<ul>
<li>standard or regular</li>
<li>fail script</li>
</ul>
The standard type of include file starts with a tilde followed by a file name.<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">~my_beer.inc</span></span><br />
<br />
<h4>
Using paths and automatic search for include files </h4>
There is an automatic search made by expect-lite to look for a simple file name (with no path defined) in the same directory as the expect-lite script, making it easy to use include files without having to worry about their location.<br />
<br />
However, if you want to keep your include files in a different location, say a common directory, that is also supported, either by specifying a relative path:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">~../common/my_beer.inc</span></span><br />
Or by an absolute path:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">~/Users/craig/common/my_beer.inc</span></span><br />
This last one may look a little confusing, since bash uses the tilde to indicate the user's home directory. This is not a bash tilde, but an expect-lite tilde, representing "include this file from this location." I have a habit of putting the extent ".inc" on my include files, which just makes them easier to find and manage. But expect-lite doesn't care about extents, you can use anything that works for you.<br />
<br />
<h4>
A Simple Example</h4>
So great, now we know how to include something, what does it actually do. In the following example, the include file, my_beer.inc will be:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">#my_beer.inc</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$beer=beer </span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">; $i bottles of $beer on the wall </span></span><br />
<br />
And the main script (example 1):<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">#!/usr/bin/env expect-lite</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"># Count the beers backwards</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$i=99</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">[ $i > 0</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"> ~my_beer.inc</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"> -$i</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">]</span></span><br />
<br />
An include script can have any expect-lite lines, and shares global variable space with the main script. In the above example,<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"> $i </span></span>is a counter variable in the while loop, but it also available in the include script which will print out the counter (using the semicolon printable comment feature).<br />
<br />
Given this simple example, it would have been just as easy to paste the include file lines into the main script and skipped the include file. And that is a natural way to look at include files. But it would miss some of the power of include files.<br />
<br />
<h4>
Include files like functions</h4>
For example, if I modified the main script just a bit, you will see that just as the <a href="http://expect-lite.blogspot.ca/2012/07/the-power-of-constants.html" target="_blank">power of constants </a>can be used on the main script when placing constants on the CLI, it can also be applied to include files.<br />
Editing the main script (example 2):<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">#!/usr/bin/env expect-lite</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"># Count the beers backwards</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$my_favourite=stout </span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$i=99</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">[ $i > 0</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"> ~my_beer.inc beer=$my_favourite</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"> -$i</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">]</span></span><br />
<br />
By adding a parameter to the include file <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">beer=$my_favourite</span></span> it is possible change the behaviour of the include file which will now print:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">99 bottles of stout on the wall</span></span><br />
<br />
But wait, you say, include files share variable space of the main script, all I had to do was set <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$beer</span></span> to <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">stout</span></span> in the main program, and this absolutely correct. But there are times when you may not want to change the value of <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$beer</span></span> in the main script, passing values to the include script allows you to use include files more like functions than just copy and pasting lines into your script. Since expect-lite does not support <i>real</i> functions, this is a method to use when desiring function-like behaviour.<br />
<br />
<h4>
Fail Script</h4>
The other type of include file is that of a fail script. The expect-lite script has no knowledge of if a step failed or passed, this is managed by expect-lite itself. A failed step will either immediately stop the script (default behaviour), or continue to the end of the script (when <a href="http://expect-lite.sourceforge.net/expect-lite_tips.html#nofail" target="_blank">*NOFAIL</a> is used).<br />
<br />
The fail script is old feature which was designed for over night regression runs, where if a script failed, some clean up may be require before running the next test script. But it has more modern uses as you will see.<br />
<br />
The fail script is declared, but no action is taken until script failure occurs (usually something expected was not returned). It is declared anywhere, usually early, in the script with the following:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">*~my_fail.inc</span></span><br />
<br />
There can only be one fail script active at a time, but the name can be changed as often as needed by declaring another fail script which will be in effect until the name is changed, or the end of the script:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">*~my_broken_bottle.inc</span></span><br />
<br />
In the following example, we determine what day it is, and it it isn't the weekend, call a fail script which sets a flag and returns to the main script (example 3):<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">#!/usr/bin/env expect-lite</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"># Check if it is the weekend, drink beer</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">*NOFAIL</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">*~no_beer.inc</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$beer_days=Saturday|Sunday </span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>date +%A</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><$beer_days</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"># use if statement to print correct beer drinking action </span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">? $weekend==no? ;no beer today :: ;hoist a beer, it is $beer_days </span></span><br />
<br />
The include fail script, which will only be run if it is not a weekend looks like:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">#no_beer.inc</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"># set flag if this script runs</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$weekend=no</span></span><br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"></span></span>With these two scripts, there is no need to know if the result of the <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">date </span></span>command fails, since the fail script will set a flag (or variable) <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$weekend </span></span>to signal to the if statement in the main script to send the correct comment (the :: means <i>else</i> in the if statement).<br />
<br />
Note the variable <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$beer_days=Saturday|Sunday </span></span> includes the regex OR (a vertical line, also called a pipe) which means <i>Saturday OR Sunday</i>. Since the expect line <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><$beer_days</span></span> is a regex evaluation, only the one word Saturday or Sunday need be found, and other days of the week will trigger the fail script. More on regex can be found at <a href="http://expect-lite.blogspot.ca/2012/09/demystify-regex-with-7-simple-terms.html" target="_blank">Dymystify Regex with 7 simple terms</a> (including OR).<br />
<br />
Of course the real power of include scripts come from reuse. For example, create a single telnet login include script, and every time a telnet is required, just call the include script. If later, you want to know what other users are on the system at the time of the login, just add a <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>w</span></span> command to the telnet login script, and now everywhere that include script is used, it will automagically list the users and what they are doing.<br />
<br />
<h4>
Including Beer</h4>
Include scripts are there to make your life easier by simple copy/paste, function calls, fail scripts, and code reuse. Automating beer drinking, include me in.<br />
<br />
<span style="font-size: x-small;">EDIT: as of version 4.7.0, include file names can not start with equals '=' or larger-than '<' as these are used for <a href="http://expect-lite.blogspot.ca/2014/04/warm-and-fuzzy.html" target="_blank"><b>fuzzy expect</b></a></span><br />
<br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-82405229538140350962014-01-06T17:51:00.001-08:002014-01-06T17:51:38.850-08:00Moving Forward<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYjEhog7SnudgLYVLN0UQL-7bTFQsjCooBQDoEsUPV8cAbV9-xb6_IQMWf8qA0K_nHwNvWcvRDScuCp9H7boTxdQfle5AoR1xso8YVhezAlUcuE_NzxjxGibI6viXb21Pq3FrqIyhRSWM/s1600/rideauvale_big_snow_snow_shoes_.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYjEhog7SnudgLYVLN0UQL-7bTFQsjCooBQDoEsUPV8cAbV9-xb6_IQMWf8qA0K_nHwNvWcvRDScuCp9H7boTxdQfle5AoR1xso8YVhezAlUcuE_NzxjxGibI6viXb21Pq3FrqIyhRSWM/s1600/rideauvale_big_snow_snow_shoes_.jpg" height="198" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Moving Forward</td></tr>
</tbody></table>
Much has happened in the past year, including 2,190 downloads of expect-lite in 2013. Thanks to comments and requests from users the following features have been added to expect-lite in 2013:<br />
<ul>
<li> Native Logging (saving output to a file) using the *LOG or *LOGAPPEND</li>
<li>Updated IF statements to apply Code Blocks to if/then <i>and</i> else blocks</li>
<li>Foreach loops using Code Blocks</li>
<li>String Math, permitting search/replace, concat, and remove strings</li>
<li>Experimental work on a web frontend, el_run.php, permitting scripts to be executed from a web browser </li>
<li>Lots of little bug fixes</li>
</ul>
With the advent of the new year, it is time to look forward, and continue to improve expect-lite always keeping in mind the mantra: <i>keeping it simple</i>. There are a few ideas under consideration for 2014 such as: range checking (e.g. 5 < $x < 10), adding environment variables such as EL_REMOTE_PORT (to allow telnet on a user defined port), and unsetting a variable (useful when using pseudo arrays).<br />
<br />
Expect-lite is user driven software. I look forward to your comments, and continuing to improve expect-lite in 2014. Moving forward while keeping it automation for the rest of us.<br />
<br />
Craig...<br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-21805792051018010952013-09-15T16:57:00.000-07:002013-09-15T16:57:10.874-07:00Code Blocks, adding structure to your script <style type="text/css">P { margin-bottom: 0.08in; }</style>
<br />
<div style="margin-bottom: 0in;">
</div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjV6Nai2AarCf8v_Tn98XKT3hgB_2b0qL-vo12ZJ8k2IMm1gy9gfQCzTD2HLLOUQxyg_XpjJyvOevLReQB9S8A2oqhzTbDL2pKczci_DujTDfFcIWnwz5rTrr-041uikYivHdgmJlxeSAE/s1600/portland_ice_oval_14in_thick_ice_sm.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjV6Nai2AarCf8v_Tn98XKT3hgB_2b0qL-vo12ZJ8k2IMm1gy9gfQCzTD2HLLOUQxyg_XpjJyvOevLReQB9S8A2oqhzTbDL2pKczci_DujTDfFcIWnwz5rTrr-041uikYivHdgmJlxeSAE/s200/portland_ice_oval_14in_thick_ice_sm.jpg" width="166" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Cold Block, I mean Code Blocks</td></tr>
</tbody></table>
<div style="margin-bottom: 0in;">
For eight long years, expect-lite was a
simple scripting language. Running scripts from the top to the bottom
of the file. At the request of an expect-lite user, code blocks were
introduced in version 4.2. Code Blocks permit grouping of lines and
add structure to your script.</div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
Code Blocks allow you to
create multi-line conditionals (if statements), while and
foreach loops. Naturally, Code Blocks can be nested. Code blocks
start with a square bracket, [, and end with a closing square
bracket, ]. Indentation within a Code Block is not required, but
makes the script more readable. Let's look at each of these in more
detail.</div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
<b>Multi-line Conditional</b></div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
It has been a limitation for a long
time, that expect-lite commands had to be on one line. So the
conditional was limited to the following, with the double colon
representing <i>else</i></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;">?if
$var > $big ? >do something :: >do something else</span></span></div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
However with code blocks, it is now
possible to insert multiple lines for the <i>then</i> and <i>else</i>
statement, such as:</div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;">?if
$var > $big ? [</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;"> ;red --- Var:$var is out of bounds</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;"> *FAIL</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;">]::[</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;"> ; === incrementing Var:$var</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;"> +$var</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;">]</span></span></div>
<div style="margin-bottom: 0in;">
In the above example, if $var is larger than $big, there is something wrong, tell the user about it using a red coloured comment line, and then mark the script as failed.</div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
<b>The While Loop</b></div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
The While loop is a simple looping
mechanism which has a test or comparison at the top of the loop, and therefore usually
needs a couple lines of setup before hand.</div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;">$i=0</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;">$max=10</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;">[
$i < $max</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;"> >echo
$i</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;"> +$i</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Courier New, monospace;"><span style="font-size: x-small;">]</span></span></div>
<div style="margin-bottom: 0in;">
In the above example, a loop counter
variable, $i, is set to zero, and a maximum count, $max, is set
before the while loop begins. As mentioned earlier, the code block
begins with the open square bracket, and the while loop test is
defined, $i < $max. The indented lines print out the value of $i
as the while loop progresses, and +$i increments the value of $i.
Lastly, the lone close square bracket on a separate line, indicates
the end of the Code Block, and hence, the while loop.</div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
<b>The Foreach Loop</b>
</div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
The foreach loop is another looping mechanism where the items to be iterated over are known beforehand. For example, the script writer may want to iterate over the planet names. In each iteration of the loop, the variable $planet will be assigned a value in the list of planets.</div>
<pre><span style="font-family: Courier New, monospace;"><span style="font-size: x-small;">[ $planet=mercury venus earth mars jupiter saturn uranus neptune</span></span></pre>
<pre><span style="font-family: Courier New, monospace;"><span style="font-size: x-small;"> >ping6 c1 $planet</span></span></pre>
<pre><span style="font-family: Courier New, monospace;"><span style="font-size: x-small;">] </span></span>
</pre>
<div style="margin-bottom: 0in;">
Of course, until the internet is extended to the interplanetary-internet, using IPv6 naturally, you may not receive an answer to the ping, but you get the idea. </div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
Foreach loops have the limitation in expect-lite, that the items in the list must be space delimited. As you can see in the example above, multiple spaces are allowed. And if your list as something else as a delimiter, such as a comma, there is an expect-lite feature called string math to help.</div>
<pre><span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">$planets=mercury,venus,earth,mars,jupiter,saturn,uranus,neptune</span></span></pre>
<pre><span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">; === use string math /search/replace/ to convert commas to spaces</span></span></pre>
<pre><span style="font-family: Courier New, monospace;"><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">=$planets /,/ /</span> </span></span></pre>
<pre>
</pre>
<div style="margin-bottom: 0in;">
Please look at the man page, or <a href="http://expect-lite.sourceforge.net/expect-lite_doc.html#string_math_variables" target="_blank">online documentation</a> for more information on string math.</div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
<b>Keeping it Simple</b></div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
Code Blocks do add a bit of complexity to the script, but the added structure will bring more power, and clarity to your scripts, all the while keeping it simple.</div>
<div style="margin-bottom: 0in;">
</div>
<pre><span style="font-family: inherit;"></span></pre>
<pre></pre>
<div style="margin-bottom: 0in;">
</div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
<br /></div>
<div style="margin-bottom: 0in;">
<br /></div>
Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com2tag:blogger.com,1999:blog-5277161037604260037.post-37429796134960991012013-07-06T06:03:00.000-07:002013-07-06T06:03:58.220-07:00Just like being thereRemote control was what expect-lite was born to do. In the early days of expect-lite, there was a need to log into an embedded linux machine and run some tests. There was also a need to log into different remote hosts with same script, so switching which remote host to test had to be easy, as easy as putting it on the command line.<br />
<br />
Of course there are several different methods to access a remote host, and to that end, expect-lite supports telnet, ssh, both will username and password options, and a third method, ssh with keys, which requires no password, but the keys have to be set up ahead of time. These methods are selected by setting an environment variable <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">EL_CONNECT_METHOD</span></span>.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEih4JVYX91j_ya-iH0a5Ahdpk620e0eUVP33A1NwxbmtoJ5DEKeHaMObCwxCHP0Ya4Nczx5vis8zHjHx0JVuXLfGFtSXQ43mzy-_-fzZ69x9z52Rfy38lxe8e4XsidnXJwoMsxHYno_LjY/s1600/miller-remote-login.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="137" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEih4JVYX91j_ya-iH0a5Ahdpk620e0eUVP33A1NwxbmtoJ5DEKeHaMObCwxCHP0Ya4Nczx5vis8zHjHx0JVuXLfGFtSXQ43mzy-_-fzZ69x9z52Rfy38lxe8e4XsidnXJwoMsxHYno_LjY/s320/miller-remote-login.png" width="320" /></a></div>
<br />
Over the years of using expect-lite I have found while handy to connect to the remote embedded system, it isn't always the best method. Often embedded systems don't have a full linux (or other OS) on them, after all it is by its nature a small system. So I have found that logging into my desktop linux system (aka localhost) provides a rich full set of utilities, and the script can then log into the embedded system (using a variable $IP to control which embedded system).<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">./myscript IP=192.168.1.10 </span></span><br />
<br />
My .expect-literc which creates my environment looks like this:<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">export EL_REMOTE_HOST=localhost<br />export EL_CONNECT_METHOD=ssh_key</span></span><br /><br />
Of course, I have set up ssh keys to my localhost, allowing expect-lite to login without using a password. This is easily done by using the configuration-only option of the installation script, install.sh, in /usr/share/doc/expect-lite (if you installed using the deb or rpm file), or in expect-lite.proj directory where ever you untarred the tarball.<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$ cd <install.sh directory></span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$ ./install.sh -c<br />=======================================<br />Installing expect-lite version "4.6.0"<br />1,2,3) ===================<br /> Configuration Only selected, skipping install steps 1,2,3 <br />...</span></span><br />
<span style="font-size: x-small;"><br /></span>
<span style="font-size: x-small;">Fine print: the install script really only sets up the environment for bash shell (or sh, or ksh). It does not set up csh (or similar shells). I use bash, and that is the default shell for most distros today. expect-lite can and does work with csh, but you will need to adjust the environment using csh syntax</span><br />
<br />
Before using the environment created by install.sh it is necessary to either a) relogin, or b) source your ~/.bashrc from the shell. Only then will the new environment take effect.<br />
<br />
With this environment, expect-lite will log into the localhost using ssh keys (yellow arrow in diagram above), and then the script will log into the remote host, such as an embedded system (the blue arrow). For additional flexibility I use the *FORK command to create another session to the localhost linux system.<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">*FORK linux</span></span><br />
And switching back to the remote host with:<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">*FORK default </span></span><br />
Now the script has access to the rich set of linux utilities, and the embedded system under test.<br />
<br />
Of course, you don't have to use this method. expect-lite supports connecting directly to the remote host as I stated earlier. Either way works well, and for the script, it is just like being there.<br />
<br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-18729720375402754682013-02-19T18:27:00.000-08:002013-02-19T18:27:59.346-08:00To quote or not to quote<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVvZo_R9RHFyqZmJBbTFhafWwVNK4fowwz_n6XVKqqvD88nTSgZkkx4k37uC2NKmwjQKmfwUUNHI22MyZpfIDXYs5x8OhwHt6IAfHRW22XMZk5J31O1uKgdVlxkpjKZqqSrGw3pPbxhzM/s1600/quote_or_not_quote.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVvZo_R9RHFyqZmJBbTFhafWwVNK4fowwz_n6XVKqqvD88nTSgZkkx4k37uC2NKmwjQKmfwUUNHI22MyZpfIDXYs5x8OhwHt6IAfHRW22XMZk5J31O1uKgdVlxkpjKZqqSrGw3pPbxhzM/s1600/quote_or_not_quote.png" /></a></div>
Quotes, really useful to convey a meaning or sentiment. When used in traditional software languages, quotes depict a string of characters. However, the mantra of expect-lite is keeping it simple, and quotes are not required.<br />
<br />
In fact, quotes aren't even optional, and can cause problems when trying to quote strings. Take the following example:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$abc="hello world"</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$def=hello world</span></span><br />
The two variables are <i>NOT</i> equal. Expect-lite will store the quote characters as part of $abc.<br />
<br />
Traditionally using quotes allows the programmer to encompass all characters between the quotes, but this becomes a problem when the quote character is part of the string to be stored. But then a whole system of escaping characters needs to be introduced.<br />
<br />
Expect-lite eliminates all that complexity by storing everything after the equal sign, spaces, quotes, tabs, everything. Even more equals signs, such as below:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$equation=5 + 2 = 7</span></span><br />
<br />
This feature is handy when assigning non-printable characters such as tab. It is possible to store the tab character by pressing the tab key after the equals sign:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$tab= </span></span><br />
<br />
The $tab can then be used to test tab completion of a command line, such as when using the bash shell, and pressing tab twice to see possible file names:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>ls /etc/rc$tab$tab</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><rc3.d</span></span><br />
<br />
Quotes are not even required when using while loops or if statements. Expect-lite will just do the right thing. For example earlier we set the variable $def, and the following will totally work:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">?if $def == hello world ? >hello back!</span></span><br />
<br />
So save the quotes for Shakespeare, and remember expect-lite is about keeping it it simple.<br />
<br />
<br />
<br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-73391083270599923162013-01-20T15:45:00.001-08:002013-01-20T15:45:38.761-08:00Turning Eight<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivfs7JFafIjn4PqjV9GPVLo1I6naO8BNPPma7A6bdKVSYxOj-C8kVRFZ-zAbLVGBlfO2HgCWvCqyf8t4prpkWIlmpja0pF0280giN5DxBTaI4wC4Sn6NK-1seJuvHf_PVnrcmGIJXTb5Y/s1600/ferry_billet_painter_8.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivfs7JFafIjn4PqjV9GPVLo1I6naO8BNPPma7A6bdKVSYxOj-C8kVRFZ-zAbLVGBlfO2HgCWvCqyf8t4prpkWIlmpja0pF0280giN5DxBTaI4wC4Sn6NK-1seJuvHf_PVnrcmGIJXTb5Y/s200/ferry_billet_painter_8.jpg" width="187" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Turning Eight</td></tr>
</tbody></table>
It started on 19 January 2005. Hard to believe it has been eight years since I first took up the keyboard and started writing expect-lite.<br />
<br />
Of course, it was a few weeks before expect-lite could actually do anything useful. But the idea of keeping it simple, and sending > and expecting < was there.<br />
<br />
In software development, as a rule, major versions don't have to be compatible with previous versions, but expect-lite has kept backward compatibility since those early days of 2005. It has been a long-standing goal to be able to run a script written years ago on the latest version of expect-lite.<br />
<ul>
<li>1.x - Basic functionality</li>
<li>2.x - Added Dynamic Variables, Better Var parsing, Embedded Expect</li>
<ul>
<li>version 2.4.0, expect-lite goes open source!</li>
</ul>
<li>3.x - Looping using %labels, multiple sessions (*FORK), break points (*INTERACT), <span style="color: #38761d;">Colour</span></li>
<li>4.x - Integrated Debugger with ^\, Integrated as a TCL library, Code Blocks [ ], User defined script help using ;;; and Logging to a file (*LOG)</li>
</ul>
Fast forward 8 years, and almost 10,000 downloads later, and expect-lite is still going strong. Many of the features added are a direct result of users' comments. For example, built in logging to a file using *LOG, has has been on the request list for a while, but I do listen, and now it is in the latest release.<br />
<br />
Thank you for all of those comments and feature requests over the years, I look forward to more.<br />
<br />
Craig...<br />
[author and maintainer of expect-lite]<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-52031070063046991122012-12-09T10:57:00.000-08:002012-12-09T10:57:35.443-08:00Writing scripts with copy and pasteSure everyone who has tried to use expect-lite knows it is easy to use if you are just using > and < in the script. But what if you want to do something a bit more tricky, say capture a value from the screen, and make a comparison.<br />
<br />
It is a bit contrived, but say for example, you want to check the number of packets received on the eth0 interface is below 100,000 (100 thousand). In expect-lite, it is quite easy to do the comparison using a <a href="http://expect-lite.sourceforge.net/expect-lite_doc.html#variables" target="_blank">dynamic variable</a>, a variable which is assigned based on the output of a command at run time. Sometimes it can be tricky to get the regex right to capture the info you want, but I will keep regex simple, and use copy and paste to solve this puzzle.<br />
<br />
<h3>
Setting up the environment</h3>
<br />
Using el_shell.elt (in the examples directory) makes it easy to try out expect-lite lines using copy and paste between two windows (an editor and terminal window). The first thing you want to do is setup two windows like this:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4eAOdQNoFKQpw6d7P2U3477gSAC_rtIDtn92FD8X2HeW3GKAe16rH3ak41Bi0GmoyUU7d4-RNbs2Y4NtvlJgYEnP_Ru1uh28xD7cUzAeK3pUlu0p_rqMyHmNQ9ArISRm3nuI8woK2mNQ/s1600/two_windows.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4eAOdQNoFKQpw6d7P2U3477gSAC_rtIDtn92FD8X2HeW3GKAe16rH3ak41Bi0GmoyUU7d4-RNbs2Y4NtvlJgYEnP_Ru1uh28xD7cUzAeK3pUlu0p_rqMyHmNQ9ArISRm3nuI8woK2mNQ/s400/two_windows.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">two window setup</td></tr>
</tbody></table>
I prefer overlapping windows, but it is up to your preference. The key is that you start an editor (I use nedit) and a terminal (or xterm) and can see them both at the same time. In this exercise, we will write lines of expect-lite code in the editor, copy them, then paste them into the terminal which is running el_shell.elt. Having overlapping windows (rather than full screen) makes it easier to see what is going on.<br />
<br />
Now that you have your editor started, go ahead and start el_shell.elt in the terminal by typing:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">./el_shell.elt</span></span><br />
<br />
el_shell.elt is pretty simple, it starts expect-lite, then does an *INTERACT to drop you into the debugger. It <i>may</i> look like you are just at the bash prompt. To confirm you are in the debugger, press ESC then h (in cygwin, press back-quote then h). You should see the following output:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">IDE: Help<br /> Key Action<br /> ---- ------<br /> <esc>s Step<br /> <esc>k sKip next step<br /> <esc>c Continue<br /> <esc>v show Vars<br /> <esc>0to9 Show N next lines of script<br /> <esc>-1to-9 Show N previous lines of script<br /> ctrl+D Quit & Exit expect-lite<br /> <esc>h this Help<br /><br />[cvmiller@fedora-arm:~]$ </span></span><br />
<br />
<h3>
Using el_shell.elt</h3>
<br />
Now that you have the environment setup, let's capture the interface RX counter. The ifconfig command will display interface counters (and other useful info). We use a simple regex, any digit, \d, with the modifier of <i>plus</i>, for one or more (see <a class="GFUQPS5BPB" href="http://expect-lite.blogspot.ca/2012/09/demystify-regex-with-7-simple-terms.html">Demystify Regex with 7 simple terms</a>). This will match a contiguous run of numbers. In the editor type the following lines:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>ifconfig eth0</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">+$eth_rx=(\d+)</span></span><br />
<br />
Select the 2 lines, copy, then switch to the terminal window, and paste the lines, the terminal output should look something like this:<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">[cvmiller@fedora-arm:~]$ ifconfig eth0<br />eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br /> inet 10.1.1.41 netmask 255.255.255.0 broadcast 10.1.1.255<br /> inet6 2001:470:1d:583:2<span style="font-size: x-small;">4</span>6:b3ff:fe0<span style="font-size: x-small;">5</span>:cbe5 prefixlen 64 scopeid 0x0<global><br /> inet6 fe80::2<span style="font-size: x-small;">4</span>6:b3ff:fe0<span style="font-size: x-small;">5</span>:cbe5 prefixlen 64 scopeid 0x20<link><br /> ether 00:<span style="font-size: x-small;">4</span>6:b3:0<span style="font-size: x-small;">5</span>:cb:e5 txqueuelen 1000 (Ethernet)<br /> RX packets 114320 bytes 24060958 (22.9 MiB)<br /> RX errors 2 dropped 80 overruns 0 frame 1<br /> TX packets 20022 bytes 4036177 (3.8 MiB)<br /> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br /><br />[cvmiller@fedora-arm:~]$ <br />Assigned Var:eth_rx=0</span></span><br /><br />
You can see that the value assigned to $eth_rx is 0 (zero). Not quite what we were looking for. Using dynamic variable assignment (aka capture), only the part inside the parenthesis will be assigned. Let's switch back to the editor and refine the dynamic assignment statement a bit more:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>ifconfig eth0</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">+$eth_rx=packets (\d+)</span></span><br />
<br />
Copy and paste (into the terminal) again. Now we see at the bottom of the terminal (removing the ifconfig output for brevity):<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">[cvmiller@fedora-arm:~]$ <br />Assigned Var:eth_rx=</span></span><span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">114320</span></span><br />
<br />
That is looking better. Let's make the script a bit more versatile by using a static variable for the interface, which could later be overridden with a constant (see <a href="http://expect-lite.blogspot.ca/2012/07/the-power-of-constants.html">The Power of Constants</a>). And suppose we want to ensure that we always assign the RX packets, we could tweak the dynamic assignment statement a bit more in the editor:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$iface=eth0</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>ifconfig $iface</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">+$eth_rx=RX packets (\d+)</span></span><br />
<br />
Copy and paste (into the terminal) again. With confidence, we see at the bottom of the terminal:<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">[cvmiller@fedora-arm:~]$ <br />Assigned Var:eth_rx=</span></span><span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">114320</span></span><br />
<br />
Now we have a RX packet value we can compare, so let's update the script in the editor a bit more, and another copy and paste into the terminal:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$rx_limit=100000</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$iface=eth0</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>ifconfig $iface</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">+$eth_rx=RX packets (\d+)</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">?if $eth_rx > <span style="font-size: x-small;">$rx_l<span style="font-size: x-small;">imit? [</span></span></span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> ;red RX packets over limit</span></span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> *FAIL</span></span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">] </span> </span></span><br />
<br />
The result now looks like:<span style="font-weight: normal;"><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"> </span></span></span><br />
<span style="font-weight: normal;"><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">Assigned Var:eth_rx=114320</span></span></span><br />
<h3>
<span style="font-weight: normal;"><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"></span></span></span></h3>
<h3>
<span style="color: #38761d;"><span style="font-weight: normal;"><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">If: expr { "<span style="font-size: x-small;">1<span style="font-size: x-small;">1</span>4320</span>" > "100000" } |then [|result=TRUE</span></span></span></span></h3>
<h3>
<span style="font-weight: normal;"><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"></span></span></span></h3>
<h3>
<span style="color: red;"><span style="font-weight: normal;"><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"> RX packets over limit</span></span></span></span></h3>
<h3>
<span style="font-weight: normal;"><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"></span></span></span></h3>
<h3>
<span style="font-weight: normal;"><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><span style="color: red;">expect-lite directive: *FAIL</span> </span></span></span></h3>
Even though the scriptlet failed, el_shell.elt is still running. This is because while in the debugger, one has immunity from failing. After all, it would not be helpful if the debugger quit just because you pasted in some code that failed.<br />
<br />
<h3>
Copy and Paste, a simple technique to create complex solutions</h3>
<br />
As you can see copying and pasting into the debugger allows you to run arbitrary expect-lite code, and see the results instantly. The debugger is not limited to simple commands, you can paste while loops, directives (like *DEBUG), include files, or other complex code and have it execute right away.<br />
<br />
The mantra of expect-lite is keeping it simple. What could be simpler than copy and pasting scriptlets while you automate complex solutions.<br />
<br />
<span style="font-size: x-small;">PS. to simplify things, I have artificially kept the RX packet counter constant, in real life, it will probably increment</span><br />
<span style="font-size: x-small;">PPS<span style="font-size: x-small;">. In Cy<span style="font-size: x-small;">gwin, <span style="font-size: x-small;">you will have <span style="font-size: x-small;">use ipconfig rather than ifconfig</span></span></span></span></span><br />
<span style="font-size: x-small;"><span style="font-size: x-small;"><span style="font-size: x-small;"><span style="font-size: x-small;"><span style="font-size: x-small;">PPP<span style="font-size: x-small;">S. In Linux/Unix/Mac you need not actually copy and paste, since X has an automatic <span style="font-size: x-small;">clipboard, by just selecting the text in the editor, moving the mouse to the terminal and pressing the middle button on the mouse (which will paste <span style="font-size: x-small;">the <span style="font-size: x-small;">s<span style="font-size: x-small;">elected text). This makes the overlapping window setup very fast.</span></span></span></span></span> </span></span></span></span></span><br />
<span style="font-size: x-small;">PPPP<span style="font-size: x-small;">S. t<span style="font-size: x-small;">he debugger <i>does</i> have a restriction on copy and paste, the first line <i>must not </i>be <span style="font-size: x-small;">indented</span></span> (lines below the first line can be indented). </span></span><br />
<span style="font-size: x-small;">PPPPP<span style="font-size: x-small;">S. el_shell.elt <span style="font-size: x-small;">was added in the examples directory <span style="font-size: x-small;">in release 4.3.3</span></span></span></span><br />
<br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-79960715885648899832012-10-17T18:11:00.000-07:002012-10-17T18:11:57.002-07:00Consuming Output, eating tables for lunchThere are times when a table is presented, and you the script writer is interested in only part of the data. The traditional method of parsing tables is to figure out the rows and columns, and assign them to variables, and check the variables for the right values. But this is a lot of work, and not in keeping with expect-lite's credo, keeping it easy.<br />
<br />
Take the following example, which can be done on any linux machine (in windows use just the route print command):<br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-size: x-small;">$ route -n<br />Kernel IP routing table<br />Destination Gateway Genmask Flags Metric Ref Use Iface<br />10.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0<br />169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 eth0<br />0.0.0.0 10.1.1.1 0.0.0.0 UG 100 0 0 eth0</span></span><br />
Without turning this into a networking lesson, you can see there are rows and columns. You will note on the last row, second column is '10.1.1.1' which is the the <i>default gateway</i> for this machine.<br />
<br />
Suppose you are interested in checking that the default gateway is correct, you could script the following:<br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">>route -n</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><10.1.1.1 </span><br />
And you would be done. You don't have to worry about which row or column the value of 10.1.1.1 is in, since expect-lite will search the entire 'route -n' output.<br />
<br />
But suppose you wanted to capture the default gateway into a <a href="http://expect-lite.sourceforge.net/expect-lite_doc.html#variables">dynamic variable</a>, and there was no guarantee that it would always be 10.1.1.1? Now things are a little more tricky, but not too much.<br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">>route -n</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><\n0.0.0.0</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">+$default_gateway=(\d+\.</span><span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">\d+\.<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">\d+\.<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">\d+)</span></span></span></span><br />
<br />
Only took 3 lines, still didn't have to know about rows and columns. The second line, '<\n0.0.0.0' is using a <i>very</i> powerful feature of expect-lite, one I call <b>consuming output</b>.<br />
<br />
When a command is sent in expect-lite, and text is returned (like the output of route -n), it is held as a big blob (a buffer) of text. With each '<' or '<<' command, it eats or consumes the blob of text when it finds a match. So in the above 3 line example, line #2 consumes all the text up to and including the 0.0.0.0 on the last line (of the route -n output). This means the first 4 lines of the table are gone, consumed. The very next IP address in the remaining text blob is '10.1.1.1', which is what is assigned to the dynamic variable $default_gateway.<br />
<br />
If you want to see text blob consumption at work, run the following:<br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">*DEBUG</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">>route -n</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><0.0.0.0</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><0.0.0.0</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><0.0.0.0 </span><br />
<br />
The output will look a bit chatty, but you will see the text blob (of route -n output) being gobbled by each '0.0.0.0'. I won't copy and paste the whole output here, but to give you and idea, it will look similar to this (abbreviated):<br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">>route -n</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">... </span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;">find<<0.0.0.0>><br /> in<<route -n<br />Kernel IP routing table<br />Destination Gateway Genmask Flags Metric Ref Use Iface<br />10.1.1.0 0.0.0.0>></span></span><span style="font-size: small;"><span style="font-family: Arial,Helvetica,sans-serif;"><br /><span style="font-family: Times,"Times New Roman",serif;">The first 4 and a half lines </span></span></span><span style="font-size: small;"><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: Times,"Times New Roman",serif;"><span style="font-size: small;"><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: Times,"Times New Roman",serif;">(including the 'route -n' line)</span></span></span> are consumed</span></span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;"><br />find<<0.0.0.0>><br /> in<< 255.255.255.0 U 0 0 0 eth0<br />169.254.0.0 0.0.0.0>><br /><br /><span style="font-family: Times,"Times New Roman",serif;"><span style="font-size: small;">The next half and a line are consumed</span></span><br /><br />find<<0.0.0.0>><br /> in<< 255.255.0.0 U 1000 0 0 eth0<br />0.0.0.0>></span></span><br />
The last chunk is consumed, leaving the next thing in the text blob, the default gateway. Each time text is found in the text blob (0.0.0.0 in this example), the top of the text blob is consumed, and no longer available to be found (or expected).<br />
<br />
Using consumption can be useful in using the <a href="http://expect-lite.sourceforge.net/expect-lite_doc.html#not_expect">not-expect feature</a> as well. As it says in the documentation, how long does one wait for something to not appear? With expect-lite it is about 100ms, or about 1/10th of a second. But what if the unexpected item comes after 1/10th of a second? You can use consumption.<br />
<br />
Suppose you never want to see the default gateway of 192.168.1.1. with the above example and using the not-expect feature, you could script the following:<br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">>route -n</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><\n0.0.0.0</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">-<192.168.1.1 </span><br />
It will only not-expect 192.168.1.1 <i>after</i> it consumes the text blob down to the last line of output.<br />
<br />
Another problem with tables is finding the right value, when multiple instances are in the table, such as 0.0.0.0 in route -n output. I use consumption to <b>bookend</b> the value I want. For example, if I want to check that the second line (the one starting with 169.254.0.0) gateway was correct, I could use consumption to find something before and after the desired value, such as:<br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">>route -n</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><169.254.0.0</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><0.0.0.0</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><255.255.0.0 </span><br />
<br />
Without having to know which row or column these items are on, it is possible to <i>expect</i> 0.0.0.0 is the gateway for 169.254.0.0. Bookending and consuming output ensures that the correct 0.0.0.0 will be matched.<br />
<br />
So go, and fear not ye tables with rows and columns, for expect-lite is here to slay and consume them, making automation quick and easy.<br />
<br />
<span style="font-size: x-small;">PS. you will remember from "<a href="http://expect-lite.blogspot.ca/2012/09/demystify-regex-with-7-simple-terms.html">Demystifying Regex</a>" that \n is a new line.</span><br />
<span style="font-size: x-small;">PPS. one could create a "simpler" regex to capture the default gateway, but then you would have to know about character classes, such as:</span><br />
<span style="font-size: x-small;">+$default_gateway=([0-9.]+)</span><br />
<span style="font-size: x-small;">PPPS. Think of a shelf with books, old school but still can be found in the library. The things holding up the books at the end of the shelf are called <a href="http://en.wikipedia.org/wiki/Bookend">bookends</a>. </span><br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-78107978051010346132012-09-14T09:18:00.000-07:002012-09-14T09:18:11.904-07:00Demystify Regex with 7 simple termsRegular Expressions aren't for everyone. Regex is a powerful cryptic conflagration of characters which mean something, if only you could figure out what.<br />
<br />
But what if there were a handful of regex terms which did 90% of what you needed. Then you could harness the immense power of regex, without having to learn a whole new language. After all, expect-lite is about making it easy.<br />
<br />
<b>Regex Whats</b><br />
<br />
Regex is made up of two parts, what to look for, and does that "thing" repeat. The <i>whats</i> are characters which describe a number, a letter or a non-printing character (such as tab). The four terms you will want to know start with a back-slash and are followed by a single letter:<br />
\d is a number<br />
\w is a letter<br />
\n is a new line (think of it as a carriage return)<br />
\t is a tab <br />
<br />
<b>Regex Repeats</b><br />
<br />
Repeats are useful for finding a string of numbers, for example 123456. Two terms for the <i>repeats</i> are:<br />
* repeats 0 or more times<br />
+ repeats 1 or more times<br />
<br />
<b>Regex matching with expect-lite</b><br />
<br />
With these six terms you can create very useful regex expressions. The following example shows the output of the route command:<br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-size: x-small;">$ route -n<br />Kernel IP routing table<br />Destination Gateway Genmask Flags Metric Ref Use Iface<br />10.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0<br />169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 eth0<br />0.0.0.0 10.1.1.1 0.0.0.0 UG 100 0 0 eth0</span></span><br />
You could use the following to match the interface:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;">>route -n</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;"><eth\d </span></span><br />
<br />
You could also match the first IP address:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;">>route -n</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;"><\d+.\d+.\d+.\d+ </span></span><br />
<br />
Using the plus, +, means regex will match any digit repeating 1 or more times. An IP addresses can have 1 to 3 digits per octet (the number between the dots). The repeat makes it easy to match a variable number of digits.<br />
<br />
Or even more useful would be to use a dynamic variable to grab the default gateway:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;">>route -n</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;"><\n0.0.0.0</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;">+$default_gw=(\d+.\d+.\d+.\d+)</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;"> </span></span><br />
By using the new line, the expect line will only match the <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">0.0.0.0</span></span> at the beginning of the line. I'll write more later about how to leverage expect-lite's capture buffer, but in this example, the <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><\n0.0.0.0</span></span> positions expect-lite to capture into a dynamic variable the very next thing that matches the regex <span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;">\d+.\d+.\d+.\d+</span></span> which in this example the value of <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$default_gw</span></span> would be <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">10.1.1.1</span></span><br />
<br />
<b>The regex OR</b><br />
<br />
The seventh term of regex that is good to know is the OR command which is the vertical line or pipe, |<br />
The pipe allows you to make a statement, such as mach this OR that. In expect-lite it would look like:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>echo "this"</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><this|that</span></span><br />
<br />
The above example is a bit contrived, but it is common to find the output of a command which might be true OR false, or enabled OR disabled, or UP or DOWN. This may be less useful for a simple match, but very useful in capturing a dynamic variable. The following command shows the interface state on a linux machine:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">$ ip link show eth0<br />2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000<br /> link/ether 00:30:65:96:b5:4a brd ff:ff:ff:ff:ff:ff</span></span><br /><br />
To capture that state of the interface (UP or DOWN) in expect-lite, it would as simple as:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;">>ip link show eth0</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;">+$intf_state=(UP|DOWN) </span></span><br />
<br />
By using a simple, yet powerful regex, it is easy to capture the states of the interface in the example above.<br />
<br />
<b>The power of 7</b><br />
<br />
I have only scratched the surface of Regex here, but it should cover 90+ percent of what you might need. The 7 simple regex terms here; the whats, the repeats, and OR, it is possible to match just about anything you need in expect-lite.<br />
<br />
<br />
<br />
<span style="font-size: x-small;">PS. The above is not entirely correct, as the dot, is also a regex expression, but the above examples will work without having to know this eighth term.</span><br />
<br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-4845948260569552862012-08-19T15:38:00.000-07:002012-08-19T15:38:11.103-07:00So you want to be root - calling sudoOver the years, I have taken several different approaches to using sudo in my scripts. When scripting expect-lite, the script either expects a response or it doesn't. sudo sometimes asks for a password, and other times it does not. Unfortunately, there is no built-in function to support sometimes.<br />
<br />
But the need to use sudo remains. It is very useful when writing scripts which need to change file ownership, permissions, insert kernel modules or running tcpdump.<br />
<br />
My latest technique of using sudo takes advantage of the sudo option '-S' which frees the script to not care if sudo requires a password.<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"># sudo password</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">$pass=secret</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># run the ID command with sudo privileges</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">>echo "$pass" | sudo -S pwd</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">>sudo id</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"><uid=0</span></span><br /><br />
The sudo option '-S' means get the password from standard in (stdin). By using the echo command, it is easy to put the password on stdin and call sudo, as you can see in the above script. You will also note, that the second time I call sudo, I do <i>not</i> use the echo-technique, since I am expecting sudo to remember the previous authentication.<br />
<br />
A note about sudo, the password which is passed to sudo, is in turn passed to the program (pwd in the above script). Not all programs take the password graciously, and in fact will give an error rather than execute as expected (tcpdump is one of these). Therefore, I find doing a seemingly non-related command like pwd is helpful in gaining authentication, and then executing the command I really want to run with sudo privileges.<br />
<br />
There is a downside to the echo-technique. The user's password will be printed on the screen. However this can be somewhat mitigated by running the script as a test user (which has sudo privileges) rather than your user id. <br />
<br />
Go ahead, be root, it is easy with sudo in expect-lite.<br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-6610551003002638692012-08-04T14:33:00.000-07:002012-08-04T14:33:15.818-07:00A tale of two scriptsOne of the cool things about expect-lite is that it has a very simple interpreter, which ignores everything it doesn't understand, rather than giving a syntax error. You can paste just about any text into an expect-lite script, such as release notes, network diagrams, or sample output from commands, and have it ignored. This feature can be exploited. <br />
<br />
In this post, I will embed a bash script inside an expect-lite script. I'll use the bash script to set up the environment, and then have the script call itself, this time using expect-lite as the interpreter. Since expect-lite ignores anything it doesn't know, it will ignore the embedded bash script.<br />
<br />
Here's the script, bash_env.elt (with 755 permissions)<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">#!/usr/bin/env bash</span><span style="font-family: "Courier New",Courier,monospace;"> </span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Little bootstrap bash script to adjust env then run expect-lite</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">export EL_CONNECT_USER=user</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">export EL_CONNECT_PASS=secret</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">export EL_REMOTE_HOST=10.1.1.15</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">export EL_CONNECT_METHOD=telnet</span><span style="font-family: "Courier New",Courier,monospace;"></span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># re-run the script as an expect-lite script</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">/usr/bin/env expect-lite c=$0 $*</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># end the bash script, and return expect-lite pass/fail</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">exit $?</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#start the expect-lite script</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">*EXP_INFO</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">@3</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">$myvar=hello world</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">$user=admin</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">>echo "$myvar"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"><world</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">></span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"></span></span><br />
Run the script:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">./bash_env.elt </span></span><br />
<br />
The script starts out like any bash script, and then sets (and exports) some environment variables which control how expect-lite connects to a remote host. Then the bash starts expect-lite feeding it the script name ($0) and any parameters that might have been typed on the command line ($*). Finally, the bash script exits (exit $?) preventing bash from reading the rest of the script.<br />
<br />
The final part of the script is expect-lite, which will be executed after expect-lite telnets to 10.1.1.15, and logs in with credentials user and secret.<br />
<br />
Of course, you don't want to hard code these parameters for every script. After all, you may use telnet as your connect method today, and change to ssh later. <br />
<br />
Embedding other scripts inside expect-lite scripts might just solve your challenging problem.<br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-15430058888210813922012-07-25T18:32:00.001-07:002012-07-25T18:32:57.402-07:00The Power of ConstantsExpect-lite has an old feature, constants, that when I put it in, I only used it occasionally. Constants are immutable, and override variables in the script. This is handy if you want to change the behaviour of the script without going in and editing the script. For example, assume you have the following script called loop.elt:<br />
<span style="font-family: "Courier New",Courier,monospace;">$max=10</span><br />
<span style="font-family: "Courier New",Courier,monospace;">$i=0</span><br />
<span style="font-family: "Courier New",Courier,monospace;">[ $i < $max</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> >echo "hello world"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> <world</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> +$i</span><br />
<span style="font-family: "Courier New",Courier,monospace;">]</span><br />
<br />
Running this script would loop send "hello world" 10 times and check for the word "world" each time. But suppose you want to stress things out by running this 10,000 times. Using a constant on the command line when you start the script will do this quickly and easily.<br />
<div style="font-family: "Courier New",Courier,monospace;">
loop.elt max=10000</div>
<br />
Note: when using a constant on the command line, don't use the dollar sign ($), it tends to confuse the shell (such as bash). <br />
<br />
Of course, you can get into trouble by trying to set a constant for $i, <b style="color: black;">don't do this!</b><br />
<div style="font-family: "Courier New",Courier,monospace;">
loop.elt i=9</div>
<br />
You will end up with an infinite loop, because $i has been defined as a constant, and will always be 9, it can <u>never</u> increment, and never be larger than $max (10). If you wait long enough, you will discover another feature of expect-lite, <i>infinite loop protection</i>. After 5000 iterations (default value), infinite loop protection will kick in and stop the script and print:<br />
<div style="color: red;">
ERROR: Infinite Loop Detected!</div>
<br />
When creating loops, another trick you may want to use is to begin your script with another variable called $start:<br />
<span style="font-family: "Courier New",Courier,monospace;">$start=0</span><br />
<span style="font-family: "Courier New",Courier,monospace;">$max=10</span><br />
<span style="font-family: "Courier New",Courier,monospace;">$i=$start</span><br />
<span style="font-family: "Courier New",Courier,monospace;">...</span><br />
<span style="font-family: "Courier New",Courier,monospace;"></span><br />
This addition allows you to define $start as a constant, now allowing control via constants of when the loop starts and when it stops (via $max). If for example, you wanted to run the script over the range 3-7, start the script as follows:<br />
<div style="font-family: "Courier New",Courier,monospace;">
loop.elt start=3 max=7</div>
<br />
In this post, I have shown how it is possible to override counting variables with constants, but constants are not limited to integers. You can override any variable in your script such as usernames and passwords, or even expected values! Anywhere a variable can be used in expect-lite, it can be overridden by a constant.<br />
<br />
Recently, I have begun to use this feature quite a bit. So much so, that I lose track of what variables are in the script, which could be used as constants. So as of version 4.3.0, I added a help feature which lists the variables in the script in "constant format" (without the dollar sign), making it easy to copy and paste into the command line. For example, typing <span style="font-family: "Courier New",Courier,monospace;">loop.elt -h </span>will print:<br />
<span style="color: #38761d; font-family: "Courier New",Courier,monospace;">Displaying assigned variables, which can be overridden with </span><br style="color: #38761d; font-family: "Courier New",Courier,monospace;" /><span style="color: #38761d; font-family: "Courier New",Courier,monospace;">a Constant on command line e.g var=value</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> i=$start</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> max=10</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> start=0</span><br /><br />
Constants make it a simple matter to override variables, making your scripts much more flexible, and powerful. <br />
<br />Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0tag:blogger.com,1999:blog-5277161037604260037.post-19544103350253700242012-07-23T16:45:00.000-07:002012-07-23T16:45:11.821-07:00Expect MoreHi All,<br />
<br />
This is a blog dedicated to making expect-lite even easier. Sure, anyone can use the simplicity of '>' and '<' in expect-lite, but there is much more.<br />
<br />
I will be writing about tips and tricks to help people get the most out of expect-lite, all the while keeping it simple. After all, expect-lite is about automation for the rest of us.<br />
<br />
Thanks for checking in, and let me know if you have ideas or topics you want to discuss.<br />
<br />
Craig...<br />
[author and maintainer of expect-lite]Craighttp://www.blogger.com/profile/15880011431496785012noreply@blogger.com0