AWS for DevOps

AWS Services:

EC2 - Elastic Compute Cloud
EBS - Elastic Block Store
EFS - Elastic File Store
S3 - Simple Storage Service
ELB
AutoScaling
VPC
IAM
EKS
ECR

AWS CLI


Cloud Service Models:

IaaS - Infrastructure as Service (Admins)

Managed by Users:
Applications
Data
Runtime
Middleware
O/S

Managed by the Vendor/Service Provider:
Virtualization
Servers
Storage
Networking

Example: EC2


PaaS -  Platform as a Service (Developers) 

Managed by Users:
Applications
Data

Managed by the Vendor/Service Provider:
Runtime
Middleware
O/S
Virtualization
Servers
Storage
Networking

Example: EKS (Elastic Kubernetes Service)


SaaS- Software as a Service 

Everything is managed by the Vendor, we just use the Softwares and pay based on the service.

Managed by the Vendor/Service Provider:
Applications
Data
Runtime
Middleware
O/S
Virtualization
Servers
Storage
Networking

Example: Cloud Watch 


AWS Global Infrastructure:

i) Availability zones (Data Centers - Physical)
ii) Geographical regions (It's a logical name, which defines the geographical location)
Example: ap-south-1, ap-southeast-1
iii) Local Region

How to decide the correct region for your project?

There are a couple of factors before deciding:
i) Customer location
ii) Price
iii) Security & Compliance Requirements
iv) Network Latency
v) Service Availability (based on whether the service available in the AWS region)


How to achieve High Availability for the application? 
By hosting the application into multiple/different Availability Zones in the same region or Multiple regions.

We can achieve the HA using AWS available Services like AutoScaling Groups, Elastic Load Balancer (ELB).

What is a Multi-clouded Application?
Applications hosted on AWS using the different cloud vendors like AWS and Azure.


EC2 (Elastic Compute Cloud) Basics

a. It's a Virtual machine on top of AWS Infrastructure (Physical Servers).
b. It provides scalable computing capacity.
c. It has flexible to create an instance based on your project requirements (like CPUs, RAM, Network,  
    Storage, Security)
d. it enables you to scale up or down to handle changes in requirements. 

EC2 Charges based on:

i) Purchasing options
ii) Instance Type and Family
iii) EBS Optimized 
iv) AMI Type (Linux or Windows)
v) Data Transfer
vi) Region

EC2 purchasing Options:

i) OnDemand Instances
a. Most flexible and expensive
b. Whenever you want you can create or terminate the servers based on your requirements. 
c. Charges are applicable only when the instance is up and running.
d. Charges are based on an hourly basis.

ii) Reserved Instances 
a. It is not flexible because, we create a server for a set of the time period like 1 year, 1 month.
b. Charges are applicable based on your reserved time whether you used it not but you need to pay for the complete reserved time period. 

iii) Spot Instances
a. It is basically used for a short amount of time like a day or a week and these unused servers are already available in the AWS. We use that kind of server only when we are doing some kind of POC and we need high configuration servers for a week or a day to test it.
b. Charges are like a contract, whoever quotes a high price for the server gets the server. 

EC2 Components: (Non-Cloud vs Ec2 Servers)

Non-Cloud Server                         EC2 Server
Operating System                         AMI  (Amazon Machine Image)
RAM                                             Instance Type (for both RAM and CPU)
CPU                                              Instance Type (for both RAM and CPU)
Hard disk (Storage)                      EBS Storage 
Network (IP Addresses)               IP Address 
Firewall                                         Security Group

EC2 Instance Family and Type:

i) General Purpose (Balance Performance)
Families: t2, m4, m3
Example: Websites, Web applications, Development. 

ii) Compute-Optimized (Compute Optimized -> high CPU Performance)
Families: c3,c4,cc2
Examples: Applications like Web-Servers, batch processing

iii) CPU-Optimized (High-end GPU)
Families: g2,p2
Examples: Amazon AppStream, Machine Learning, High-Performance databases, Vidoe encoding.

iii) Memory-Optimized (Large RAM)
Families: r3, r4, x1,cr1
Examples: In-memory databases, data mining 

iv) Storage Optimized
Families: d2,i2,i3,hi1,hs1
Examples: NAS, Data warehouse, NoSQL

Different type of AMIs:
i) Community AMIs: It basically comes with Operating Systems.
ii) MarketPlace AMIs: It comes with Operating System with additional licensing software. 
iii) My AMIs: You can create your own AMI with the required Operating System and Additional Software and settings.


Some points to be remembered for EC2:
-> By default, every Ec2 instance has private IP allocated. 
-> We can create or launch Ec2 instances with or without Public IP.
-> Private IP addresses used only for instance communications.

What is the eligible/limit of EBS storage for free tier EC2 instances?
Every Month is 30G. It doesn't matter you create a single instance or many instances but you are eligible for a max 30G.

What is a User-data option in AWS while creating an instance?
Options while creating an Ec2 instance:

Choose AMI
Choose Instance Type
Configure Instance
Add Storage
Add Tags
Configure Security Group
Review and Complete




Elastic IP vs Public IP?
-> Public IP is the non-static IP and whenever you stop and restart the server public will change.
-> Elastic IP is the static IP and it's consistent whenever the server restarts.
-> Each Elastic IP, it's chargeable based on hourly-based usage and it's chargeable only when we don't use it after allocated and it's not chargeable if we start using it after allocation. However, for Public IP it's free. 


Storage Services:

EBS
EFS
S3


General Storage types:

Block Storage:  The data and file will be maintained in equally distributed blocks. 

Advantages:
Provide better performance, less latency, and suitable for databases and high-performance applications.

ASW Service for Block Storage is "EBS (Elastic Block Store)".

File Storage: It's like a traditional file system. It will maintain files as a single entity and will not break the files into multiple entities.

ASW Service for File Storage is "EFS (Elastic File Store)".

Object Storage: It's used to maintain data like unstructured data. Each Object (File/Data) will get a unique identifier and Endpoint (URL), from anywhere you can access the data like browser, URL and you don't need to mount like Google Drive, Dropbox. 

ASW Service for Object Storage is "S3 (Simple Storage Service)"


A snapshot in AWS:
Snapshot is generally a backup of the storage device. 

Benefits:
i) It is used to recover the lost data.
ii) It is used to  Migration the server from one region to another.
iii)If we want to migrate the data or server from one region to another region we use snapshot.

Create snapshot -> copy the snapshot to another region -> Create a Volume from the snapshot -> attach the volume by mounting directory (logical name)

Important points to be remembered:
Both EC2 instance and EBS volume should present in the same availability zone (region).


Steps to attach additional Volume to EC2 instance:
1- Create volume by selecting a volume option under EBS
2- Select the required Availability zone.
3- If required Add tags to the volume (Optional).
4- After creating the new Volume the status should be shown as "Available" 
4- Attach the volume 


Can we attach one EBS to multiple EC2 instances simultaneously?
NO, not possible simultaneously. If required you can detach it from the current server and attach it to the required server.

How to list block Storage type?
# lsblk


EBS (Elastic Block Storage)


Steps to add additional disk to the Ec2 instance:

Step-1: Create volume

EBS -> Volumes -> Create Volumes -> Select Size and Availability Zone -> Create Volume

Step-2: Attach volume to the instance

EBS -> Volumes -> Select which volume to be attached -> Actions -> Attach Volume -> Select Instance ->  Attach

Step-3: Validate the disk is visible on the server

# lsblk

Step-4: Format the attached disk because the disk is a raw disk

# mkfs.ext4 /dev/xvdf

Step-5; Mount the disk with the required directory name on the server

# mkdir -p /opt/jenkins
# chown ec2-user:ec2user /opt/jenkins
# mount /dev/xvdf /opt/jenkins
# df -h


How to take Snapshot of the Volume?
EBS -> Snapshots -> Create Snapshot -> Select volume which you want to backup ->  Create Snapshot

How to take Snapshot of the Instance?
EBS -> Snapshots -> Create Snapshot -> Select instance which you want to backup ->  Create Snapshot

How to automate/schedule to create/take snapshot?
Using Lifecycle Manager.

EBS -> Lifecycel Manager -> Create Lifecycle Policy -> Policy type -> Select resource type -> Choose IAM role -> Select Policy Schedule -> Select further options based on the requirement 

How to migrate the server or disk from one availability zone to another availability zone in same region?

Step-1: Create a Snapshot of the server which you want to migrate
EBS -> Snapshots -> Create Snapshot -> Select volume which you want to backup ->  Create Snapshot
Step-2: Create a new instance on the region where you want to migrate
Step-3: Create a volume from the snapshot 
EBS -> Snapshots -> Actions -> Create Volume -> Select Availability zone ->  Create Volume
Step-4: Attache the volume to the instance
EBS -> Volume -> Actions -> Attach Volume -> Select Instance -> Attach
Step-5: Mount the volume on the server
# mount /dev/svdf  /jenkins
# df -h

Note: Make sure not to format the mounted disk because it's not a raw disk it has data and if you you formt the data will be lost.

How to migrate the server or disk from one availability zone to another availability zone in different region?

Step-1: Create a Snapshot of the server which you want to migrate
EBS -> Snapshots -> Create Snapshot -> Select volume which you want to backup ->  Create Snapshot
Step2: Create a copy from the snapshot
EBS -> Snapshot -> Select Volume -> Action -> Copy -> Select the Destination Region -> Copy 
Step-3: Switch to the Destination region
Step-4: Create a volume from the snapshot 
EBS -> Snapshots -> Actions -> Create Volume -> Select Availability zone ->  Create Volume
Step-5: Attache the volume to the instance
EBS -> Volume -> Actions -> Attach Volume -> Select Instance -> Attach
Step-6: Mount the volume on the server
# mount /dev/svdf  /jenkins
# df -h


What is the maximum EBS volume/size capacity?
16 TB

Can we attach snapshot directory to the instance?
no, you first need to create volume and then attach the volume to the instance

How to recover Ec2 instance ssh access if we lost pem file?

Comments

Post a Comment

Popular posts from this blog

Auditing in Oracle database

Unified Auditing: Oracle Database 12c Release 1 introduces a new auditing feature called Unified Auditing (12.1). Oracle Unified Auditing modifies the database's fundamental auditing capabilities. In earlier versions of Oracle, each component has its own audit trail. With unified auditing, all auditing is compiled into a single repository and view. Due to the fact that all audit data is now in a single location and a single format, there is a double simplification as a result. This data is made available in a consistent format in the UNIFIED AUDIT TRAIL data dictionary view, which is made possible by the unified audit trail, which is stored in a read-only table in the AUDSYS schema in the SYSAUX tablespace.  DBAs can build audit rules and then assign them to various users using Oracle 12c Unified Auditing, which has the same advantages as giving users varying system capabilities based on their responsibilities. The unified_audit_trail database will contain all of the audit records....
Read more

rs.stepDown() in MongoDB replication

It instructs the primary of the replica set to become a secondary. After the primary steps down,the eligible secondaries will hold an election for primary. Generally, the rs.stepDown() does not immediately step down the primary. If no electable secondaries are up to date with the primary, the primary waits up to secondaryCatchUpPeriodSecs (by default 10 seconds)  for a secondary to catch up. Once an electable secondary is available, the method steps down the priamry.  Once stepped down, the original primary becomes a secondary and is ineligible from becoming primary again for the remainder of time specified by stepDownSecs. The rs.stepDown() method attempts to terminate long-running user's operations that block the primary from stepping down, such as an index build, a write operation, or a map-reduce job. Syntax: rs.stepDown(stepDownSecs, secondaryCatchUpPeriodSecs)
Read more

Tuples in Python

 Tuples Basically Tuples in Python are very similar to Lists, but, Tuples are immutable which means that they can not be modified/changed. We generally use Tuples to make sure the values are not changed like, dates, Year, and days of the week or dates on a Calendar.  Creating a Tuple: >>> my_t1=(1,2,3) >>> my_t2=('one','two','three') >>> my_t1 (1, 2, 3) >>> my_t2 ('one', 'two', 'three') -- check the length of tupple: >>> len(my_t1) 3 >>> len(my_t2) 3 -- Tuples can have mixed objects as like Lists. >>> my_t3=(1,'two',3,'four') >>> my_t3 (1, 'two', 3, 'four') -- indexing with tuples  >>> my_t3=(1,'two',3,'four') >>> my_t3[2] 3 Immutable Tuples We can not modify the value. Here is the example: >>> my_t3=(1,'two',3,'four') >>> my_t3[2]=4 Traceback (most recent call last):  ...
Read more