Python Automate
Python Menu Program Integrating Different Technologies
Why python for automation?🤩
Python is an easy-to-learn programming language that allows organizations to script custom automation and reap the time-savings.
As we all know python is a versatile, easy to use and fast to develop. We have so many libraries in python that make our task easy and fast.
The best way to learn any language is to implement concepts and functionalities in code. As we come across a new concept we will try to implement it with python, and that is exactly what we are trying to do here
We will add our new functionalities as we learn as a new menu which will make our code more readable and easily accessible
We are going to provide following functionalities✨:
1.Run RHEL8 Commands
2. Online Python Interpreter
3. Setup Hadoop Cluster
4. Create a Partition
5. AWS-CLI used
6. Docker Technology Set-up
7. Configure Web Server
8. Open Browser
9. Go for SSH Login
Let’s go in depth now🤓
📌Run RHEL8 Commands
🔅Following are the redhat commands we will run using python
1.date
2. calendar
3. check IP address
4. Check Internet Connectivity
5. create a file
6. create New user
Isn’t it fascinating how we can run one OS command from another OS
Python make it happen, The OS module in python provides functions for interacting with the operating system. This module provides a portable way of using operating system dependent functionality.
os.system() method execute the command (a string) in a subshell. This method is implemented by calling the Standard C function system(), and has the same limitations. If command generates any output, it is sent to the interpreter standard output stream. Whenever this method is used then the respective shell of the Operating system is opened and the command is executed on it.
//for date command
os.system("date")//for calculator
os.system("cal")//To find ip address
os.system("ifconfig enp0s3")//To check internet connectivity
os.system("ping goo.gl -c 4")//To create a file
i = input("write file name: ")
os.system("gedit {}".format(i))//To create new user
i = input("type username: ")
os.system("useradd {}".format(i))
os.system("passwd {}".format(i))
📌Online python interpreter
🔅Using this menu we are trying to interact with python interpreter
This is done using same os.system() function, it helps us to download python3 if its not already installed in our system
//open python interpreter
os.system("python3")📌Setup Hadoop Cluster
Let’s first take some brief about what is Hadoop cluster
Apache Hadoop is an open source, Java-based, software framework and parallel data processing engine. It enables big data analytics processing tasks to be broken down into smaller tasks that can be performed in parallel by using an algorithm (like the MapReduce algorithm), and distributing them across a Hadoop cluster.
A Hadoop cluster is a collection of computers, known as nodes, that are networked together to perform these kinds of parallel computations on big data sets.
🔅Let’s see how can we configure Hadoop cluster using python, we are providing following functionalities for Hadoop cluster setup
1.Check Hadoop Software Install or Version of Hadoop
2. Make a System as a Node (Name/Data/Client)
3. Start this Node (Name/Data)
4. Check the Service is Start
5. Check the Report of HDFS
6. Upload File in HDFS
7. Check the folder in HDFS
8. Read the File from HDFS
9. Back in main menu
Check version of hadoop or see if it is installed or not
//check hadoop version
os.system("hadoop -version")Configure NameNode
To configure a system as a name node we have to make a directory which will keep all metadate about our hadoop cluster and configure two files which are: 1]hdfs-ste.xml and 2]core-site.xml
//Make a System as a Node (Name/Data/Client)
//Configure System as namenode
f = input("Type folder name: ")
os.system("mkdir /{}".format(f))
print(" Your /{} is Created...".format(f))
os.system("vim /etc/hadoop/hdfs-site.xml")
os.system("vim /etc/hadoop/core-site.xml")Format NameNode
os.system("hadoop namenode -format")Configure Data Node
To configure a system as a datanode we have to make a directory which will participate to store data for our hadoop cluster and configure two files which are: 1]hdfs-ste.xml and 2]core-site.xml
f = input("Type folder name: ")
os.system("mkdir /{}".format(f))
print(" Your /{} is Created...".format(f))
print("...Open hdfs-site.xml file...")
os.system("vim /etc/hadoop/hdfs-site.xml")
os.system("vim /etc/hadoop/core-site.xml")Configure Client Node
os.system("vim /etc/hadoop/core-site.xml")After configuring each node for our cluster we will have to start NameNode and DataNode
os.system("hadoop-daemon.sh start namenode")
os.system("hadoop-daemon.sh start datanode")Check the Service is Start
To check if we have successfully started our data node or client node following command is used
os.system("jps")Now Let’s see if we have added all nodes in our cluster successfully or not, following command will report number of nodes in our cluster along with the memory available for our cluster
os.system("hadoop dfsadmin -report")Now that we have setup our Hadoop cluster successfully lets try to put files in our cluster and read file from cluster
//Put file in hadoop cluster
f = input("Type the Location/name of file: ")
os.system("hadoop fs -put {}".format(f))//Gives all the files present in our hadoop cluster
os.system("hadoop fs -ls /")//Read the file from cluster
f = input("Type which file you want to read: ")
os.system("hadoop fs -cat /{}".format(f))
📌Create a Partition
Partitioning allows us to divide your hard drive into isolated sections, where each section behaves as its own hard drive.
🔅We have two options available:
1] for LVM (Logical volume Partition) Partition
2] for Increase or Decrease the size of Static Partiton
1]LVM allows for very flexible disk space management. It provides features like the ability to add disk space to a logical volume and its filesystem while that filesystem is mounted and active and it allows for the collection of multiple physical hard drives and partitions into a single volume group which can then be divided into logical volumes.
Let’s see how it’s done
Before creating partition we will have to check if we have hard disk attached or not, if not you will have to add a hard disk mannually
a = input("Confirm Your System have Attached New Hard Disk [Y/N]? ")
b = "Y"
if a != b:
print("First you ADD new Hard Disk")
exit()After successfully adding HDD let’s start partitioning
🔅Following are the menus which will help us creating LVM partion successfully
1: to see device name
2: to see description of Physical volume
3: Create Physical Volume
4: to see description of Volume Group
5: Create Volume Group
6: to see description of Logical Volume
7: Create Logical Volume
8: Format the partition with (ext4) type
9: Attach/Linked the partition with folder
10: Check my device is linked with folder
//see device name
os.system("fdisk -l")//describe physical volume
os.system("pvdisplay")//create physical volume
d = input("Write your device name: ")
os.system("pvcreate {}".format(d))//describe volume group
os.system("vgdisplay")//create volume group
g = input("Write Group name: ")
d = input("Write your 1st device name: ")
d2 = input("Write your 2nd device name: ")
os.system("vgcreate {0} {1} {2}".format(g,d,d2))//describe logical volume
os.system("lvdisplay")//Create logical volume
s = input("Give the Size of Logic Volume with (K,M,G,T): ")
n = input("Write Name of your Logic Volme: ")
os.system("lvcreate --size {0} --name {1}".format(s,n))//Format the partition with (ext4) type
f = input("Write Device name which is you want to format: ")
os.system("mkfs.ext4 {}")
print("this device is formatted with (ext4) type...Done")//Attach/Linked the partition with folder
m = input("Write device name which is You want to attached/linked: ")
f = input("Write folder name (where you want to linked): ")
os.system("mount {0} {1}".format(m,f))//Check if device is linked with folder
os.system("df -h")
2] for Increase or Decrease the size of Static Partiton
🔅In order to make our static partition dynamic so that we can increase or decrease our partition size we have to integrate the concept of LVM with the partitions
1. to see report of File System and Disk
2. for Increase/Decrease the size
3. for Formatting the Increasing part
4. for Linking the Partition
//to see report of File System and Disk
os.system("lsblk")
os.system("df -h")
os.system("tput setaf 6")
print("\tNote in which Partition you want to Increase or Decrease the size, also note the mountpoint/folder name")//for Increase/Decrease the size
d = input("Type Disk name: ")
os.system("umount {}".format(d))
os.system("fdisk {}".format(d))//for Formatting the Increasing part
p = input("Type Partition disk name for format : ")
os.system("e2fsck -f {}".format(p))
os.system("resize2fs {}".format(p))//for Linking the Partition
p = input("Type partition disk name for linking: ")
m = input("Type the folder name for linking: ")
os.system("mount {0} {1}".format(p,m))
📌AWS-CLI
The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.
🔅we are implementing following functionalities for AWS-CLI using python
1.Check AWS-CLI Install
2. AWS Help
3. AWS Configure/Authentication process
4. Check Descriptions of Instances
5. Launch Amazon-Linux Instance on AWS Cloud
6. Stop Instances
7. Start Instances
//Check AWS-CLI Install
os.system("aws --version")//AWS Help
os.system("aws help")//AWS Configure/Authentication process
os.system("aws configure")//Check Descriptions of Instances
os.system("aws ec2 describe-instances")//Launch Amazon-Linux Instance on AWS Cloud
a = input("How many instances you want to launch: ")
os.system("aws ec2 run-instances --image-id ami-0e306788ff2473ccb --instance-type t2.micro --count {0} --key-name alinux".format(a))//Stop Instances
a = input("Type Instance Id: ")
os.system("aws ec2 stop-instances --instance-ids {}".format(a))//Start Instance
a = input("Type Instance Id: ")
os.system("aws ec2 start-instances --instance-ids {}".format(a))
📌Docker Technology Set-up
What is docker?
Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications.
A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another. A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.
🔅Here are some functionalities for docker using python
1.Check Docker install or not
2. If not, Then Install Docker
3. Start the docker service
4. Check Information about Docker
5. Show How many Docker Image have
6. Download Docker Image
7. Check How many Container (O.S.) are running or stopped
8. Launch New Container (O.S.)
9. Go Inside the Container (O.S.)
10. Shut down Container (O.S.)
11. Start the Container (0.S.)
//Docker Version(if installed)
os.system("docker version")//If not, Then Install Docker
os.system("yum install docker-ce --nobest")//Start the docker service
os.system("systemctl start docker")//Check Information about Docker
os.system("docker info")//Show How many Docker Image have
os.system("docker images")//Download Docker Image
print("First, Search the OS Image ....")
s = input("Write Image name for searching: ")
os.system("docker search {}".format(s))
print("Now, Go for Download the Image....")
d = input("Write the Image name: ")
v = input("Type Image Version name: ")
os.system("docker pull {0}:{1}".format(d,v))//Check How many Container (O.S.) are running or stopped
os.system("docker ps -a")//Launch New Container (O.S.)
n = input("Type Container name: ")
m = input("Type Image name: ")
v = input("Type Image Version name: ")
os.system("docker run -dit --name {0} {1}:{2}".format(n,m,v))//Go Inside the Container (O.S.)
n = input("Type Container name: ")
os.system("docker attach {}".format(n))//Shut down Container (O.S.)
n = input("Type Container name: ")
os.system("docker stop {}".format(n))//Start the Container (0.S.)
n = input("Type Container name: ")
os.system("docker start {}".format(n))
📌Configure Web Server
A web server is a computer that runs websites. It’s a computer program that distributes web pages as they are requisitioned. The basic objective of the web server is to store, process and deliver web pages to the users. This intercommunication is done using Hypertext Transfer Protocol (HTTP).
🔅Let’s automate web server configuration…
1.Check the Web Server Software Install or Not
2. If not, Then Install Web Server Software
3. Check status, Web Service Start or Not
4. If not. Then Start the Web Service
5. Create File which is you want to show the Client
6. Check list of how many files we have
7. Check this file on Web Browser
//Check the Web Server Software Install or Not
os.system("yum list httpd")//If not, Then Install Web Server Software
os.system("yum install httpd")//Check status, Web Service Start or Not
os.system("systemctl status httpd")//start service
os.system("systemctl start httpd")//Create File which is you want to show the Client
f = input("Type File name: ")
os.system("gedit {}".format(f))//Check list of how many files we have
os.system("ls /var/www/html")//Check this file on Web Browser
f = input("Type which file you want to Show: ")
print(" wait ....browser is open..")
import webbrowser
webbrowser.open("http://192.168.43.223/{}".format(f))
📌Open Browser
🔅How can we open different sites like google, facebook gmail and many more using python? Let’s see
1.Open Google Search on Browser
2. Open Facebook page for Login
3. Open Gmail page for Login
4. Open Docker-Hub page
5. Open AWS on browser
The webbrowser module provides a high-level interface to allow displaying Web-based documents to users. Under most circumstances, simply calling the open() function from this module will do the right thing.
//Opne google
webbrowser.open("https://www.google.com/")//open facebook
webbrowser.open("https://www.facebook.com/")//open gmail
webbrowser.open("google.com%2Fmail%2F&osid=1&service=mail&ss=1<mpl=default&rm=false")//open docker-hub
webbrowser.open("https://hub.docker.com/")//open AWS
webbrowser.open("https://aws.amazon.com/")
📌Go for SSH Login
Remote Login is a process in which user can login into remote site i.e. computer and use services that are available on the remote computer.
An SSH client allows you to connect to a remote computer running an SSH server. The Secure Shell (SSH) protocol is often used for remote terminal connections, allowing you to access a text-mode terminal on a remote computer as if you were sitting of it.
🔅we are performing two operations:
1.for Go to Login in Remote System
2. for Only Execute Some Command in Remote System
//Go to Login in Remote System
ip = input("type IP Address of the Remote system: ")
os.system("ssh {} -y".format(ip))//Execute Some Command in Remote System
ip = input("type IP Address of the Remote system: ")
c = input("Type the Command name: ")
os.system("ssh {0} -y {1}".format(ip,c))
CONCLUSION :
we integrated python with different technologies like AWS, Docker, Hadoop, SSH and many more….
Call us python freak but integrating different technologies with python is such a fun thing to do, you get to learn a lot and you can truly see the power of python
This is just the start let’s keep exploring
I hope you found this blog helpful and it interested you more towards python….your feedback is valuable, clap if you like it👏❤️
