mirror of
https://github.com/adambard/learnxinyminutes-docs.git
synced 2024-12-23 17:41:41 +00:00
318 lines
8.6 KiB
Markdown
318 lines
8.6 KiB
Markdown
---
|
|
category: framework
|
|
name: ShutIt
|
|
contributors:
|
|
- ["Ian Miell", "http://ian.meirionconsulting.tk"]
|
|
filename: learnshutit.py
|
|
---
|
|
|
|
## ShutIt
|
|
|
|
ShutIt is an shell automation framework designed to be easy to use.
|
|
|
|
It is a wrapper around a Python-based expect clone (pexpect).
|
|
|
|
You can look at it as 'expect without the pain'.
|
|
|
|
It is available as a pip install.
|
|
|
|
## Hello World
|
|
|
|
Starting with the simplest example. Create a file called example.py:
|
|
|
|
```python
|
|
import shutit
|
|
session = shutit.create_session('bash')
|
|
session.send('echo Hello World', echo=True)
|
|
```
|
|
|
|
Running this with:
|
|
|
|
```bash
|
|
python example.py
|
|
```
|
|
|
|
outputs:
|
|
|
|
```bash
|
|
$ python example.py
|
|
echo "Hello World"
|
|
echo "Hello World"
|
|
Hello World
|
|
Ians-MacBook-Air.local:ORIGIN_ENV:RhuebR2T#
|
|
```
|
|
|
|
The first argument to 'send' is the command you want to run. The 'echo'
|
|
argument outputs the terminal interactions. By default ShutIt is silent.
|
|
|
|
'send' takes care of all the messing around with prompts and 'expects' that
|
|
you might be familiar with from expect.
|
|
|
|
|
|
## Log Into a Server
|
|
|
|
Let's say you want to log into a server and run a command. Change example.py
|
|
to:
|
|
|
|
```python
|
|
import shutit
|
|
session = shutit.create_session('bash')
|
|
session.login('ssh you@example.com', user='you', password='mypassword')
|
|
session.send('hostname', echo=True)
|
|
session.logout()
|
|
```
|
|
|
|
which will log you into your server (if you replace with your details) and
|
|
output the hostname.
|
|
|
|
```
|
|
$ python example.py
|
|
hostname
|
|
hostname
|
|
example.com
|
|
example.com:cgoIsdVv:heDa77HB#
|
|
```
|
|
|
|
Obviously that's insecure! Instead you can run:
|
|
|
|
```python
|
|
import shutit
|
|
session = shutit.create_session('bash')
|
|
password = session.get_input('', ispass=True)
|
|
session.login('ssh you@example.com', user='you', password=password)
|
|
session.send('hostname', echo=True)
|
|
session.logout()
|
|
```
|
|
|
|
which forces you to input the password:
|
|
|
|
```
|
|
$ python example.py
|
|
Input Secret:
|
|
hostname
|
|
hostname
|
|
example.com
|
|
example.com:cgoIsdVv:heDa77HB#
|
|
```
|
|
|
|
Again, the 'login' method handles the changing prompt from a login. You give
|
|
ShutIt the login command, the user you expect to log in as, and a password
|
|
(if needed), and ShutIt takes care of the rest.
|
|
|
|
'logout' handles the ending of a 'login', handling any changes to the prompt
|
|
for you.
|
|
|
|
## Log Into Multiple Servers
|
|
|
|
Let's say you have a server farm of two servers, and want to log onto both.
|
|
Just create two sessions and run similar login and send commands:
|
|
|
|
```python
|
|
import shutit
|
|
session1 = shutit.create_session('bash')
|
|
session2 = shutit.create_session('bash')
|
|
password1 = session1.get_input('Password for server1', ispass=True)
|
|
password2 = session2.get_input('Password for server2', ispass=True)
|
|
session1.login('ssh you@one.example.com', user='you', password=password1)
|
|
session2.login('ssh you@two.example.com', user='you', password=password2)
|
|
session1.send('hostname', echo=True)
|
|
session2.send('hostname', echo=True)
|
|
session1.logout()
|
|
session2.logout()
|
|
```
|
|
|
|
would output:
|
|
|
|
```bash
|
|
$ python example.py
|
|
Password for server1
|
|
Input Secret:
|
|
|
|
Password for server2
|
|
Input Secret:
|
|
hostname
|
|
hostname
|
|
one.example.com
|
|
one.example.com:Fnh2pyFj:qkrsmUNs# hostname
|
|
hostname
|
|
two.example.com
|
|
two.example.com:Gl2lldEo:D3FavQjA#
|
|
```
|
|
|
|
## Example: Monitor Multiple Servers
|
|
|
|
We can turn the above into a simple monitoring tool by adding some logic to
|
|
examine the output of a command:
|
|
|
|
```python
|
|
import shutit
|
|
capacity_command="""df / | awk '{print $5}' | tail -1 | sed s/[^0-9]//"""
|
|
session1 = shutit.create_session('bash')
|
|
session2 = shutit.create_session('bash')
|
|
password1 = session.get_input('Password for server1', ispass=True)
|
|
password2 = session.get_input('Password for server2', ispass=True)
|
|
session1.login('ssh you@one.example.com', user='you', password=password1)
|
|
session2.login('ssh you@two.example.com', user='you', password=password2)
|
|
capacity = session1.send_and_get_output(capacity_command)
|
|
if int(capacity) < 10:
|
|
print('RUNNING OUT OF SPACE ON server1!')
|
|
capacity = session2.send_and_get_output(capacity_command)
|
|
if int(capacity) < 10:
|
|
print('RUNNING OUT OF SPACE ON server2!')
|
|
session1.logout()
|
|
session2.logout()
|
|
```
|
|
|
|
Here you use the 'send\_and\_get\_output' method to retrieve the output of the
|
|
capacity command (df).
|
|
|
|
There are much more elegant ways to do the above (e.g. have a dictionary of the
|
|
servers to iterate over), but it's up to you how clever you need the Python to
|
|
be.
|
|
|
|
|
|
## More Intricate IO - Expecting
|
|
|
|
Let's say you have an interaction with an interactive command line application
|
|
you want to automate. Here we will use telnet as a trivial example:
|
|
|
|
```python
|
|
import shutit
|
|
session = shutit.create_session('bash')
|
|
session.send('telnet', expect='elnet>', echo=True)
|
|
session.send('open google.com 80', expect='scape character', echo=True)
|
|
session.send('GET /', echo=True, check_exit=False)
|
|
session.logout()
|
|
```
|
|
|
|
Note the 'expect' argument. You only need to give a subset of telnet's
|
|
prompt to match and continue.
|
|
|
|
Note also the 'check\_exit' argument in the above, which is new. We'll come back
|
|
to that. The output of the above is:
|
|
|
|
```bash
|
|
$ python example.py
|
|
telnet
|
|
telnet> open google.com 80
|
|
Trying 216.58.214.14...
|
|
Connected to google.com.
|
|
Escape character is '^]'.
|
|
GET /
|
|
HTTP/1.0 302 Found
|
|
Cache-Control: private
|
|
Content-Type: text/html; charset=UTF-8
|
|
Referrer-Policy: no-referrer
|
|
Location: http://www.google.co.uk/?gfe_rd=cr&ei=huczWcj3GfTW8gfq0paQDA
|
|
Content-Length: 261
|
|
Date: Sun, 04 Jun 2017 10:57:10 GMT
|
|
|
|
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
|
|
<TITLE>302 Moved</TITLE></HEAD><BODY>
|
|
<H1>302 Moved</H1>
|
|
The document has moved
|
|
<A HREF="http://www.google.co.uk/?gfe_rd=cr&ei=huczWcj3GfTW8gfq0paQDA">
|
|
here
|
|
</A>.
|
|
</BODY></HTML>
|
|
Connection closed by foreign host.
|
|
```
|
|
|
|
Now back to 'check\_exit=False'. Since the telnet command returns a failure exit
|
|
code (1) and we don't want the script to fail, you set 'check\_exit=False' to
|
|
let ShutIt know you don't care about the exit code.
|
|
|
|
If you didn't pass that argument in, ShutIt gives you an interactive terminal
|
|
if there is a terminal to communicate with. This is called a 'pause point'.
|
|
|
|
|
|
## Pause Points
|
|
|
|
You can trigger a 'pause point' at any point by calling
|
|
|
|
```python
|
|
[...]
|
|
session.pause_point('This is a pause point')
|
|
[...]
|
|
```
|
|
|
|
within your script, and then continue with the script by hitting CTRL and ']'
|
|
at the same time. This is great for debugging: add a pause point, have a look
|
|
around, then continue. Try this:
|
|
|
|
```python
|
|
import shutit
|
|
session = shutit.create_session('bash')
|
|
session.pause_point('Have a look around!')
|
|
session.send('echo "Did you enjoy your pause point?"', echo=True)
|
|
```
|
|
|
|
with output like this:
|
|
|
|
```bash
|
|
$ python example.py
|
|
Have a look around!
|
|
|
|
Ians-Air.home:ORIGIN_ENV:I00LA1Mq# bash
|
|
imiell@Ians-Air:/space/git/shutit ⑂ master +
|
|
CTRL-] caught, continuing with run...
|
|
2017-06-05 15:12:33,577 INFO: Sending: exit
|
|
2017-06-05 15:12:33,633 INFO: Output (squashed): exitexitIans-Air.home:ORIGIN_ENV:I00LA1Mq# [...]
|
|
echo "Did you enjoy your pause point?"
|
|
echo "Did you enjoy your pause point?"
|
|
Did you enjoy your pause point?
|
|
Ians-Air.home:ORIGIN_ENV:I00LA1Mq#
|
|
```
|
|
|
|
|
|
## More Intricate IO - Backgrounding
|
|
|
|
Returning to our 'monitoring multiple servers' example, let's imagine we
|
|
have a long-running task that we want to run on each server. By default, ShutIt
|
|
works serially which would take a long time. But we can run tasks in the
|
|
background to speed things up.
|
|
|
|
Here you can try an example with the trivial command: 'sleep 60'.
|
|
|
|
|
|
```python
|
|
import shutit
|
|
import time
|
|
long_command="""sleep 60"""
|
|
session1 = shutit.create_session('bash')
|
|
session2 = shutit.create_session('bash')
|
|
password1 = session1.get_input('Password for server1', ispass=True)
|
|
password2 = session2.get_input('Password for server2', ispass=True)
|
|
session1.login('ssh you@one.example.com', user='you', password=password1)
|
|
session2.login('ssh you@two.example.com', user='you', password=password2)
|
|
start = time.time()
|
|
session1.send(long_command, background=True)
|
|
session2.send(long_command, background=True)
|
|
print('That took: ' + str(time.time() - start) + ' seconds to fire')
|
|
session1.wait()
|
|
session2.wait()
|
|
print('That took: ' + str(time.time() - start) + ' seconds to complete')
|
|
```
|
|
|
|
My laptop says it took 0.5 seconds to run fire those two commands, and then just
|
|
over a minute to complete (using the 'wait' method).
|
|
|
|
Again, this is trivial, but imagine you have hundreds of servers to manage like
|
|
this and you can see the power it can bring in a few lines of code and one
|
|
Python import.
|
|
|
|
|
|
## Learn More
|
|
|
|
There's a lot more that can be done with ShutIt.
|
|
|
|
To learn more, see:
|
|
|
|
[ShutIt](https://ianmiell.github.io/shutit/)
|
|
[GitHub](https://github.com/ianmiell/shutit/blob/master/README.md)
|
|
|
|
It's a broader automation framework, and the above is its 'standalone mode'.
|
|
|
|
Feedback, feature requests, 'how do I?'s highly appreciated! Reach me at
|
|
[@ianmiell](https://twitter.com/ianmiell)
|