VPC per envrionvment versus Single VPC for all environments

This blog talks about the two possible ways of hosting your infrastructure in Cloud, though it will be more close to hosting on AWS as it is a real life example but this problem can be applied to any cloud infrastructure set-up. I'm just sharing my thoughts and pros & cons of both approaches but I would love to hear from the people reading this blog about their take as well what do they think.


Before jumping right away into the real talk I would like to give a bit of background on how I come up with this blog, I was working with a client in managing his cloud infrastructure where we had 4 environments dev, QA, Pre Production and Production and each environment had close to 20 instances, apart from applications instances there were some admin instances as well such as Icinga for monitoring, logstash for consolidating logs, Graphite Server to view the logs, VPN server to manage access of people.

At this point we got into a discussion that whether the current infrastructure set-up is the right one where we are having a separate VPC per environment or the ideal setup would have been a single VPC and the environments could have been separated by subnet's i.e a pair of subnet(public private) for each environment

Both approaches had some pros & cons associated with them

Single VPC set-up

Pros:

1. You only have a single VPC to manage
2. You can consolidate your admin app's such as Icinga, VPN server.

Cons:

1. As you are separating your environments through subnets you need granular access control at your subnet level i.e instances in staging environment should not be allowed to talk to dev environment instances. Similarly you have to control access of people at granular level as well
2. Scope of human error is high as all the instances will be on same VPC.

VPC per environment setup

Pros:

1. You have a clear separation between your environments due to separate VPC's.
2. You will have finer access control on your environment as the access rules for VPC will effectively be access rules for your environments.
3. As an admin it gives you a clear picture of your environments and you have an option to clone you complete environment very easily.

Cons:

1. As mentioned in pros of Single VPC setup you are at some financial loss as you would be duplicating admin application's across environments

In my opinion the decision of choosing a specific set-up largely depends on the scale of your environment if you have a small or even medium sized environment then you can have your infrastructure set-up as "All environments in single VPC", in case of large set-up I strongly believe that VPC per environment set-up is the way to go.

Let me know your thoughts and also the points in favour or against of both of these approaches.

Chef Solo an Introduction

Introduction

Chef Solo is simple way to begin working with Chef. It is an open source version of the chef-client that allows using cookbooks with nodes without requiring access to a server. Chef Solo runs locally and requires that a cookbook (and any of its dependencies) be on the same physical disk as the node. It is a limited-functionality version of the chef-client and does not support the following:

• Node data storage
• Search indexes
• Centralized distribution of cookbooks
• A centralized API that interacts with and integrates infrastructure components
• Authentication or authorization
• Persistent attributes  

Installing chef-client  (Pre-requisite : curl )
Login to your box and run the following command to install the chef. Make sure that curl program is available on your box.

 curl -L https://www.opscode.com/chef/install.sh | bash  

To check if the installation was successful check the version of the installed chef-solo by:

 chef-solo -v  

                                  

Making Chef Repository

Next step is to setup a file structure that will help organize various Chef files. Opscode, the makers of Chef provide one sample structure. They call it simply the Chef Repository.

 wget http://github.com/opscode/chef-repo/tarball/master  

 

 tar zxf master 

 mv opscode-chef-repo-**** chef-repo/ 

Assign cookbook's path to the newly created cookbook directory inside the Chef Repository which will hold the cookbook

  mkdir .chef  
  echo "cookbook_path ['/root/chef-repo/cookbooks' ]" > .chef/knife.rb   
  knife cookbook site download apt  

.Chef folder

For Chef Solo this directory generally contains only knife.rb file. A knife.rb file is used to specify the chef-repo-specific configuration details for Knife. This file is the default configuration file and is loaded every time this executable is run. The configuration file is located at: ~/.chef/knife.rb. If a

knife.rb file is present in the . chef/knife.rb directory in the chef-repo, the settings contained within that file will override the default configuration settings. Sample content of knife.rb file can be:

 cookbook_path [ '/root/chef-repo/cookbooks' ]  
 role_path [ '/root/chef-repo/roles' ]  
 environment_path [ ' /root/chef-repo/environments ' ]  
 data_bag_path [ ' /root/chef-repo/data_bags ' ]  

Getting Started with Chef Solo

Before we're able to run Chef Solo on our servers, we will need to add two files to our local Chef repository: solo.rb and node.json.

The solo.rb file tells Chef Solo where to find the cookbooks, roles, and data bags.

The node.json file sets the run list (and any other node-specific attributes if required).

 Create a solo.rb file inside our Chef repository with the following contents:

 current_dir = File.expand_path(File.dirname(__FILE__))  
 file_cache_path "#{current_dir}"  
 cookbook_path "#{current_dir}/cookbooks"  
 role_path "#{current_dir}/roles"  
 data_bag_path "#{current_dir}/data_bags"  

Create a file called node.json inside your Chef repository with the following contents:

 {  
      "run_list": [ "recipe[<recipe-name>]" ]  
 }  

            

Example:- 
In this example i am going to install apt cookbook and the recipe which i am going to use is apt and here is my solo.rb and node.json files looks like

Our first Chef run  

Goto chef-repo folder and execute following command

 chef-solo -c solo.rb -j node.json  

How it works:

1. solo.rb configures Chef Solo to look for its cookbooks, roles, and data bags   inside the current directory: the Chef repository.

     2. Chef Solo takes its node configuration from a JSON file, in our example we simply        called it node.json. If we're going to manage multiple servers, we'll need a separate    &nbsp &nbsp &nbsp file.

3. Then, Chef Solo just executes a Chef run based on the configuration data found in

           solo.rb and node.json