<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://galnug.com/feed.xml" rel="self" type="application/atom+xml" /><link href="https://galnug.com/" rel="alternate" type="text/html" /><updated>2026-06-19T09:37:42+07:00</updated><id>https://galnug.com/feed.xml</id><title type="html">Nugroho</title><subtitle>Writing about Cloud and Data Engineering, Linux, and Kubernetes, with a strong focus on AWS. Sometimes I explore blockchain, crypto, and finance, mixing technical notes with real-world insights.</subtitle><author><name>Nugroho</name></author><entry><title type="html">Monitoring Metrics Datadog</title><link href="https://galnug.com/monitoring-l1-datadog/" rel="alternate" type="text/html" title="Monitoring Metrics Datadog" /><published>2026-03-15T21:00:00+07:00</published><updated>2026-03-15T21:00:00+07:00</updated><id>https://galnug.com/monitoring-l1-datadog</id><content type="html" xml:base="https://galnug.com/monitoring-l1-datadog/"><![CDATA[<p>it’s just for monitoring.</p>

<h2 id="1-eks-monitoring">1. EKS Monitoring</h2>
<h2 id="ec2-monitoring">EC2 Monitoring</h2>

<h2 id="references">References</h2>
<p>Datadog Doc: <a href="https://docs.datadoghq.com/">Documentation</a></p>]]></content><author><name>Nugroho</name></author><category term="cloud" /><category term="aws" /><category term="datadog" /><summary type="html"><![CDATA[it’s just for monitoring.]]></summary></entry><entry><title type="html">Monitoring Metrics AWS</title><link href="https://galnug.com/monitoring-l1-aws/" rel="alternate" type="text/html" title="Monitoring Metrics AWS" /><published>2026-03-14T21:00:00+07:00</published><updated>2026-03-14T21:00:00+07:00</updated><id>https://galnug.com/monitoring-l1-aws</id><content type="html" xml:base="https://galnug.com/monitoring-l1-aws/"><![CDATA[<p>it’s just for monitoring.</p>

<h2 id="1-amazon-web-services">1. Amazon Web Services</h2>

<h2 id="amazon-ec2">Amazon EC2</h2>
<h3 id="cpuutilization">CPUUtilization</h3>
<p>The percentage of allocated EC2 compute units currently in use. This metric indicates how heavily the instance CPU is being utilized.</p>
<ul>
  <li>Unit: Percent (%)</li>
  <li>Statistic: Average, Maximum</li>
</ul>

<p>Example</p>
<ul>
  <li>Average CPU utilization remains above 80% for 15 minutes, indicating the instance may be undersized.</li>
  <li>CPU utilization remains below 10% for several days, suggesting potential overprovisioning.</li>
</ul>

<h3 id="statuscheckfailed">StatusCheckFailed</h3>
<p>Reports whether the instance has passed all status checks in the last minute.</p>

<p>This metric can be either 0 (passed) or 1 (failed).</p>
<ul>
  <li>Unit: Count</li>
  <li>Statistic: Maximum</li>
</ul>

<h3 id="status">Status</h3>

<h2 id="monitoring-explanation">Monitoring Explanation</h2>

<p>If we configure:</p>
<ul>
  <li>Metric: CPUUtilization</li>
  <li>Time range: Last 7 days</li>
  <li>Statistic: Average</li>
  <li>Period: 1 hour</li>
</ul>

<p>it means:
For each 1-hour interval during the last 7 days, CloudWatch calculates the average CPU utilization of all data points collected within that hour.</p>

<p>Suppose between 09:00 and 10:00 CloudWatch collected these CPU values:</p>

<table>
  <thead>
    <tr>
      <th>Time</th>
      <th>CPUUtilization</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>09:00</td>
      <td>20%</td>
    </tr>
    <tr>
      <td>09:05</td>
      <td>30%</td>
    </tr>
    <tr>
      <td>09:10</td>
      <td>40%</td>
    </tr>
    <tr>
      <td>09:15</td>
      <td>50%</td>
    </tr>
    <tr>
      <td>…</td>
      <td>…</td>
    </tr>
  </tbody>
</table>

<p>The average for that 1-hour period might be:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">(</span>20 + 30 + 40 + 50 + ...<span class="o">)</span> / number of samples
<span class="o">=</span> 35%
</code></pre></div></div>

<p>For a <strong>7-day range</strong>, we will get: <code class="language-plaintext highlighter-rouge">7 days × 24 hours = 168 data points</code>.</p>]]></content><author><name>Nugroho</name></author><category term="cloud" /><category term="aws" /><category term="datadog" /><category term="grafana" /><summary type="html"><![CDATA[it’s just for monitoring.]]></summary></entry><entry><title type="html">Kubernetes Notes</title><link href="https://galnug.com/kubernetes-notes/" rel="alternate" type="text/html" title="Kubernetes Notes" /><published>2026-02-10T21:00:00+07:00</published><updated>2026-02-10T21:00:00+07:00</updated><id>https://galnug.com/kubernetes-notes</id><content type="html" xml:base="https://galnug.com/kubernetes-notes/"><![CDATA[<p>it’s just kubernetes notes.</p>

<h2 id="1-the-background">1. The Background</h2>
<p>Before Kubernetes existed, running containers in production was much more manual and operationally heavy.</p>

<p>In the early days of container adoption, an engineer used simple commands like:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run app
docker run nginx

</code></pre></div></div>

<figure>
  
<img src="/assets/images/post/kube-notes/1.png" alt="Foo" />

  <figcaption>Running containers in the early days</figcaption>
</figure>

<p>This worked well for small experiments or single-server environments. But once applications needed to run in production, there are few problems started to appear.</p>

<h3 id="11-manual-scaling">1.1 Manual Scaling</h3>
<p>When traffic increased, engineers had to manually start more containers.</p>

<figure>
  
<img src="/assets/images/post/kube-notes/2.png" alt="Foo" />

  <figcaption>Can't scale manually</figcaption>
</figure>

<p>That meant:</p>
<ul>
  <li>Finding a server with enough CPU and memory</li>
  <li>Running new containers manually</li>
  <li>Keeping track of where everything was running</li>
</ul>

<p>There was no intelligent system deciding:</p>
<ul>
  <li>Where the container should run</li>
  <li>How many replicas were needed</li>
  <li>When to scale up or scale down</li>
</ul>

<p>Everything required human intervention.</p>

<h3 id="12-no-built-in-self-healing-lack-of-fault-tolerance">1.2 No Built-in Self-Healing (Lack of Fault Tolerance)</h3>

<p>If a container crashed due to:</p>
<ul>
  <li>Application bugs</li>
  <li>Memory issues</li>
  <li>Node failure</li>
</ul>

<p>It would simply stop running.</p>

<figure>
  
<img src="/assets/images/post/kube-notes/3.png" alt="Foo" />

  <figcaption>Lack of Fault Tolerance</figcaption>
</figure>

<p>There was no automatic mechanism to:</p>
<ul>
  <li>Detect failure</li>
  <li>Restart the container</li>
  <li>Replace it on another healthy machine</li>
</ul>

<p>Engineers had to monitor and fix it manually.</p>

<h3 id="13-operational-complexity-at-scale">1.3 Operational Complexity at Scale</h3>

<p>Running 3–5 containers manually is manageable.</p>

<p>Running 300 containers across multiple servers? That becomes a serious operational problem.</p>

<p>The Engineers needed a system that could:</p>
<ul>
  <li>Automatically schedule containers</li>
  <li>Handle scaling</li>
  <li>Restart failed workloads</li>
  <li>Manage networking</li>
  <li>Maintain desired state</li>
</ul>

<h2 id="2-container-orchestration">2. Container Orchestration</h2>
<p>Container orchestration automates the deployment, management, scaling, and networking of containers. Kubernetes is one of them.</p>

<figure>
  
<img src="/assets/images/post/kube-notes/4.png" alt="Foo" />

  <figcaption>Kubernetes Container Orchestration</figcaption>
</figure>

<p>Kubernetes then:</p>
<ul>
  <li>Finds the best nodes</li>
  <li>Sarts the containers</li>
  <li>Monitors them</li>
  <li>Restarts them if they fail</li>
  <li>Scales them when needed</li>
</ul>

<p>In short:</p>

<p>Docker runs containers.</p>

<p>Kubernetes runs containerized systems.</p>

<h3 id="21-architecture-of-kubernetes">2.1 Architecture of Kubernetes</h3>
<p>A Kubernetes cluster consists of a control plane + a set of worker machines, called nodes, that run containerized applications.</p>

<figure>
  
<img src="/assets/images/post/kube-notes/5.svg" alt="Foo" />

  <figcaption>Kubernetes Architecture</figcaption>
</figure>

<h3 id="22-workflow">2.2 Workflow</h3>
<p>User communicates to Control Plane and provides necessary instructions to run containerized applications.</p>

<figure>
  
<img src="/assets/images/post/kube-notes/6.jpg" alt="Foo" />

  <figcaption>Basic worfkflow of kubernetes</figcaption>
</figure>

<p>If you have instructed Kubernetes to run 1 container of Apache, Kubernetes will launch it in one of the worker nodes and will  regularly monitor the state of that container to ensure it always runs.</p>

<p>There are three ways to connect to the cluster, through API, CLI, and GUI.</p>

<p>But, i prefer using CLI since it’s faster for most operation.</p>

<h2 id="3-kubectl-and-authentication">3. kubectl and authentication</h2>

<p>To connect using kubectl to your cluster, you need file called kube config. You can refer to the documentation for this.</p>

<ul>
  <li><a href="https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html">Install kubectl</a></li>
  <li><a href="https://docs.aws.amazon.com/eks/latest/userguide/create-kubeconfig.html">Configure kube config</a></li>
</ul>

<p>To see config file run:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># View Merged Kubeconfig Settings</span>
kubectl config view
<span class="nb">cat</span> ~/.kube/config

<span class="c"># View only the current context (This tells you which cluster your commands are currently targeting)</span>
kubectl config current-context

<span class="c"># List all available contexts (This shows all the defined clusters you can switch between)</span>
kubectl config get-contexts

<span class="c"># List all defined clusters</span>
kubectl config get-clusters

</code></pre></div></div>

<h2 id="4-kubernetes-pods">4. Kubernetes Pods</h2>
<p>A Pod is the smallest deployable unit in Kubernetes.</p>

<p>Each Pod:</p>

<ul>
  <li>Contains one or more containers</li>
  <li>Shares one IP address</li>
  <li>Can share volumes (storage)</li>
  <li>Runs on one Node (EC2)</li>
</ul>

<p>There are few characteristics of a pods:</p>

<h3 id="51-shared-network">5.1 Shared Network</h3>
<p>All containers inside a Pod:</p>
<ul>
  <li>Share the same IP address</li>
  <li>Share the same port space</li>
  <li>Can communicate via localhost</li>
</ul>

<p>For example, if Container A runs on port 3000, Container B can call:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>http://localhost:3000
</code></pre></div></div>
<h3 id="52-shared-storage">5.2 Shared Storage</h3>
<p>Containers inside a Pod can share volumes.</p>

<p>Example use case:</p>
<ul>
  <li>One container writes logs</li>
  <li>Another container processes logs</li>
</ul>

<h3 id="53-ephemeral-temporary">5.3 Ephemeral (Temporary)</h3>
<p>Pods are not permanent.</p>

<p>If a Pod:</p>
<ul>
  <li>Crashes</li>
  <li>Deleted</li>
  <li>Node fails</li>
</ul>

<p>Kubernetes creates a new Pod (with a new IP).</p>

<p>That’s why never rely on:</p>
<ul>
  <li>Pod IP</li>
  <li>Local container storage</li>
</ul>

<h2 id="5-create-kubernetes-objects">5. Create Kubernetes Objects</h2>
<p>There are two main ways to create objects in Kubernetes:</p>
<ol>
  <li>Using kubectl run (command line)</li>
  <li>Using a manifest file (YAML file)</li>
</ol>

<h3 id="51-kubectl-run">5.1 kubectl run</h3>
<p><code class="language-plaintext highlighter-rouge">kubectl run</code> is a quick way to create a Pod from the command line.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl run nginx-pod <span class="nt">--image</span><span class="o">=</span>nginx

<span class="c"># Check with </span>
kubectl get pods
</code></pre></div></div>

<p>What this does:</p>
<ul>
  <li>Creates a Pod</li>
  <li>Pulls the nginx image</li>
  <li>Runs it inside the cluster</li>
</ul>

<p>It’s fast and simple, but this approach is not good for production.</p>

<h3 id="52-manifest-file">5.2 Manifest File</h3>
<p>A manifest file is a configuration file written in YAML. Instead of typing everything in the terminal, you define your object in a file.</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">nginx-pod</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">containers</span><span class="pi">:</span>
    <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx-container</span>
      <span class="na">image</span><span class="pi">:</span> <span class="s">nginx</span>
      <span class="na">ports</span><span class="pi">:</span>
        <span class="pi">-</span> <span class="na">containerPort</span><span class="pi">:</span> <span class="m">80</span>
</code></pre></div></div>

<p>Apply with <code class="language-plaintext highlighter-rouge">kubectl apply -f pod.yaml</code>.</p>

<p>This approach is easy to manage and can be stored in Git.</p>

<h2 id="6-structure-of-manifest-file">6. Structure of Manifest File</h2>
<p>A manifest file has three main sections:</p>
<ol>
  <li><code class="language-plaintext highlighter-rouge">apiVersion</code>: Specifies the Kubernetes API version to use.</li>
  <li><code class="language-plaintext highlighter-rouge">kind</code>: Specifies the type of Kubernetes object (e.g., Pod, Deployment, Service).</li>
  <li><code class="language-plaintext highlighter-rouge">metadata</code>: Contains metadata about the object, such as its name and labels.</li>
  <li><code class="language-plaintext highlighter-rouge">spec</code>: Contains the specification of the desired state of the object.</li>
</ol>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">apiVersion</span><span class="pi">]</span>
<span class="na">kind</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">Resource Type</span><span class="pi">]</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">Resource Name</span><span class="pi">]</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="pi">[</span><span class="nv">Resource-specific Configuration</span><span class="pi">]</span>
</code></pre></div></div>

<h3 id="61-generating-manifest-file-through-cli-command">6.1 Generating Manifest File through CLI Command</h3>
<p>You can generate a manifest file using <code class="language-plaintext highlighter-rouge">kubectl</code> with the <code class="language-plaintext highlighter-rouge">--dry-run</code> and <code class="language-plaintext highlighter-rouge">-o yaml</code> flags.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl run nginx-pod <span class="nt">--image</span><span class="o">=</span>nginx <span class="nt">--dry-run</span><span class="o">=</span>client <span class="nt">-o</span> yaml <span class="o">&gt;</span> pod.yaml
</code></pre></div></div>
<p>This command:</p>
<ul>
  <li>Creates a Pod named <code class="language-plaintext highlighter-rouge">nginx-pod</code> with the nginx image</li>
  <li>Does not actually create the Pod (<code class="language-plaintext highlighter-rouge">--dry-run=client</code>)</li>
  <li>Outputs the manifest in YAML format (<code class="language-plaintext highlighter-rouge">-o yaml</code>)</li>
  <li>Saves it to a file named <code class="language-plaintext highlighter-rouge">pod.yaml</code></li>
</ul>

<h2 id="7-multi-container-pods">7. Multi-Container Pods</h2>
<p>A Pod can contain multiple containers that work together. They share the same network, namespace and storage.</p>

<p>For example, you can have:</p>
<ul>
  <li>An application container</li>
  <li>A logging container that processes logs from the application container</li>
</ul>

<p>This allows us to run multiple related processes together in the same Pod.</p>

<h3 id="71-multi-container-pods-configuration">7.1 Multi-Container Pods Configuration</h3>
<p>Here’s an example of a Pod with two containers:</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">multi-container-pod</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">containers</span><span class="pi">:</span>
    <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">app-container</span>
      <span class="na">image</span><span class="pi">:</span> <span class="s">my-app-image</span>
      <span class="na">ports</span><span class="pi">:</span>
        <span class="pi">-</span> <span class="na">containerPort</span><span class="pi">:</span> <span class="m">80</span>
    <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">log-processor</span>
      <span class="na">image</span><span class="pi">:</span> <span class="s">log-processor-image</span>
</code></pre></div></div>
<p>In this example:</p>
<ul>
  <li>The <code class="language-plaintext highlighter-rouge">app-container</code> runs the main application</li>
  <li>The <code class="language-plaintext highlighter-rouge">log-processor</code> container processes logs from the application container
Both containers share the same network and can communicate via localhost.</li>
</ul>

<h3 id="72-exec-into-container">7.2 Exec into Container</h3>
<p>We can use <code class="language-plaintext highlighter-rouge">kubectl exec</code> to run commands inside a container in a Pod.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl <span class="nb">exec</span> <span class="nt">-it</span> multi-container-pod <span class="nt">--</span> /bin/bash
</code></pre></div></div>

<p>This is default when run <code class="language-plaintext highlighter-rouge">kubectl exec</code> it will connect to the first container. To connect into other containers in Pod, run below command:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl <span class="nb">exec</span> <span class="nt">-it</span> multi-container-pod <span class="nt">-c</span> log-processor <span class="nt">--</span> /bin/bash
</code></pre></div></div>

<p>Just add <code class="language-plaintext highlighter-rouge">-c</code> flag with <code class="language-plaintext highlighter-rouge">&lt;container-name&gt;</code>.</p>

<h2 id="8-labels-and-selectors">8. Labels and Selectors</h2>
<p>Labels are key-value pairs attached to Kubernetes objects.</p>

<p>They are used to:</p>
<ul>
  <li>Organize resources</li>
  <li>Group related objects</li>
  <li>Select specific objects.</li>
</ul>

<p>Labels can be added to:</p>
<ul>
  <li>Pods</li>
  <li>Nodes</li>
  <li>Deployments</li>
  <li>Services</li>
  <li>Any Kubernetes object</li>
</ul>

<p>Example:</p>

<p>We can label a node like this:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl label nodes worker-node-1 <span class="nb">env</span><span class="o">=</span>production
</code></pre></div></div>

<p>Or inside a manifest file:</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">metadata</span><span class="pi">:</span>
  <span class="na">labels</span><span class="pi">:</span>
    <span class="na">env</span><span class="pi">:</span> <span class="s">production</span>
    <span class="na">tier</span><span class="pi">:</span> <span class="s">backend</span>
</code></pre></div></div>

<p>Here:</p>
<ul>
  <li><code class="language-plaintext highlighter-rouge">env</code> is the key</li>
  <li><code class="language-plaintext highlighter-rouge">production</code> is the value</li>
</ul>

<p>For Selector, Selectors are used to find objects with specific labels.</p>

<p>For example, a Service can select Pods with:</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">selector</span><span class="pi">:</span>
  <span class="na">app</span><span class="pi">:</span> <span class="s">frontend</span>
</code></pre></div></div>

<p>So this means: 
<code class="language-plaintext highlighter-rouge">Connect to all Pods that have label app=frontend.</code></p>

<p><strong>Simple Example</strong></p>

<p>Imagine we have 10 Pods.</p>

<p>Some have:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">labels</span><span class="pi">:</span>
  <span class="na">app</span><span class="pi">:</span> <span class="s">frontend</span>
</code></pre></div></div>

<p>Others have:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">labels</span><span class="pi">:</span>
  <span class="na">app</span><span class="pi">:</span> <span class="s">backend</span>
</code></pre></div></div>

<p>A selector with <code class="language-plaintext highlighter-rouge">app=frontend</code> will only target the frontend Pods.</p>

<h2 id="9-replicaset">9. ReplicaSet</h2>
<p>Let’s say we have requirement to run 5 Pods based on Nginx image all the time.</p>

<p>Of course, first we have to run it manually. But there are some issues when we run it with manual step.</p>

<ul>
  <li>If  a pod failed, let’s say because of node failure, process crash, etc. Kubernetes will not automatically recreate it. We would have to manually detect the failure and recreate the pod.</li>
  <li>And if we need to scale to more or fewer pods, we would have to manually create or delete pod definition and apply the changes.</li>
</ul>

<p>So, this problem can be solved by <strong>Replica Set</strong>.</p>

<p>A ReplicaSet is a Kubernetes resource that ensures a specific number of Pod replicas are always running.</p>

<p>In simple terms:</p>

<blockquote>
  <p>A ReplicaSet makes sure the number of Pods we want is always Running.</p>
</blockquote>

<p>So if a Pod crashes or is deleted, the ReplicaSet will automatically create a new Pod to replace it.</p>

<h3 id="91-why-replicaset-is-useful">9.1 Why ReplicaSet is Useful</h3>
<p>Applications need to stay available. ReplicaSet helps by:</p>
<ul>
  <li>Keeping the desired number of Pods running</li>
  <li>Automatically replacing failed Pods</li>
  <li>Supporting scaling (more or fewer Pods)</li>
</ul>

<p>Example:</p>

<p>If we want 3 Pods running and one fails, Kubernetes will create a new Pod so the total stays 3.</p>

<p>Here is an example of ReplicaSets Manifest:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">apps/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ReplicaSet</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">nginx-replicaset</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">replicas</span><span class="pi">:</span> <span class="m">3</span>
  <span class="na">selector</span><span class="pi">:</span>
    <span class="na">matchLabels</span><span class="pi">:</span>
      <span class="na">app</span><span class="pi">:</span> <span class="s">nginx</span>
  <span class="na">template</span><span class="pi">:</span>
    <span class="na">metadata</span><span class="pi">:</span>
      <span class="na">labels</span><span class="pi">:</span>
        <span class="na">app</span><span class="pi">:</span> <span class="s">nginx</span>
    <span class="na">spec</span><span class="pi">:</span>
      <span class="na">containers</span><span class="pi">:</span>
        <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
          <span class="na">image</span><span class="pi">:</span> <span class="s">nginx</span>
</code></pre></div></div>

<p>Explanation:</p>
<ul>
  <li><code class="language-plaintext highlighter-rouge">replicas: 3</code> → Kubernetes keeps 3 Pods running</li>
  <li><code class="language-plaintext highlighter-rouge">selector</code> → Finds Pods with label <code class="language-plaintext highlighter-rouge">app: nginx</code></li>
  <li><code class="language-plaintext highlighter-rouge">template</code> → Defines how new Pods should be created</li>
</ul>

<table>
  <thead>
    <tr>
      <th>Pod</th>
      <th>ReplicaSet</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Runs a single instance</td>
      <td>Manages multiple Pods</td>
    </tr>
    <tr>
      <td>No self-healing</td>
      <td>Automatically replaces failed Pods</td>
    </tr>
    <tr>
      <td>Manual scaling</td>
      <td>Automatic scaling</td>
    </tr>
  </tbody>
</table>

<h3 id="92-challange-with-replicasets">9.2 Challange with ReplicaSets</h3>
<p>ReplicaSets are useful for maintaining the number of running Pods, but they have several limitations:</p>

<h4 id="921-no-automatic-updates">9.2.1 No automatic updates</h4>
<p>ReplicaSets only maintain the number of <strong>Running Pods</strong>.</p>

<p>If we update the Pod template (for example changing the image from <code class="language-plaintext highlighter-rouge">nginx</code> to <code class="language-plaintext highlighter-rouge">redis</code>), <strong>existing Pods are not updated automatically</strong>.</p>

<p>We would have to manually recreate the Pods.</p>

<h4 id="922-no-rollback-mechanism">9.2.2 No rollback mechanism</h4>
<p>If a configuration change causes an error, ReplicaSet cannot easily revert to the previous version.</p>

<p>This makes updates risky in production environments.</p>

<h4 id="923-label-collision-problem">9.2.3 Label collision problem</h4>
<p>ReplicaSets use <strong>label selectors</strong> to manage Pods.</p>

<p>If a ReplicaSet selector matches Pods created by another resource, the ReplicaSet may <strong>start managing those Pods even though it didn’t create them</strong>.</p>

<p>This can cause unexpected behavior.</p>

<p>In real-world Kubernetes usage, we usually do not create ReplicaSets directly.</p>

<p>Instead, we create a Deployment, which automatically manages ReplicaSets for us.</p>

<h2 id="10-deployment">10. Deployment</h2>
<p>A Deployment is a Kubernetes resource used to manage and update applications running in Pods.</p>

<p>It manages ReplicaSets and provides additional features such as updates, rollbacks, and better application management.</p>

<p>In simple terms:</p>

<blockquote>
  <p>A Deployment manages ReplicaSets and provides a safe way to run and update applications in Kubernetes.</p>
</blockquote>

<p>Deployments provide features that ReplicaSets lack:</p>
<ul>
  <li>Rolling updates to update Pods gradually</li>
  <li>Rollback to previous versions if updates fail and it had versioning</li>
  <li>Better management of ReplicaSets</li>
  <li>Safer updates for production applications</li>
</ul>

<p>When we create a Deployment, Kubernetes automatically creates and manages a ReplicaSet. Because it is built on top of ReplicaSets.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Deployment
   ↓
ReplicaSet
   ↓
Pods
</code></pre></div></div>

<p>The Deployment handles updates and scaling, while the ReplicaSet maintains the number of Pods.</p>

<p>Example:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">apps/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Deployment</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">nginx-deployment</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">replicas</span><span class="pi">:</span> <span class="m">3</span>
  <span class="na">selector</span><span class="pi">:</span>
    <span class="na">matchLabels</span><span class="pi">:</span>
      <span class="na">app</span><span class="pi">:</span> <span class="s">nginx</span>
  <span class="na">template</span><span class="pi">:</span>
    <span class="na">metadata</span><span class="pi">:</span>
      <span class="na">labels</span><span class="pi">:</span>
        <span class="na">app</span><span class="pi">:</span> <span class="s">nginx</span>
    <span class="na">spec</span><span class="pi">:</span>
      <span class="na">containers</span><span class="pi">:</span>
        <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
          <span class="na">image</span><span class="pi">:</span> <span class="s">nginx</span>
</code></pre></div></div>

<p>This Deployment will:</p>
<ul>
  <li>Run 3 Pods</li>
  <li>Use the nginx container</li>
  <li>Automatically manage updates and scaling.</li>
</ul>

<h3 id="101-multiple-replicasets-in-deployment">10.1 Multiple ReplicaSets in Deployment</h3>
<p>When we update a Deployment (for example changing the container image), Kubernetes creates a new ReplicaSet for the new version.</p>

<p>This means a Deployment can have multiple ReplicaSets, where:</p>
<ul>
  <li>The old ReplicaSet contains the previous version of Pods</li>
  <li>The new ReplicaSet contains the updated Pods</li>
</ul>

<p>Kubernetes gradually shifts traffic from the old Pods to the new ones.</p>

<p>This process is called a <strong>Rolling Update</strong>.</p>

<p>Example flow:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Deployment
   ├── ReplicaSet <span class="o">(</span>old version<span class="o">)</span>
   │      └── Pods <span class="o">(</span>nginx<span class="o">)</span>
   └── ReplicaSet <span class="o">(</span>new version<span class="o">)</span>
          └── Pods <span class="o">(</span>nginx:1.25<span class="o">)</span>
</code></pre></div></div>

<p>During the update:</p>
<ul>
  <li>New Pods are created</li>
  <li>Old Pods are slowly removed</li>
  <li>The application stays available with <strong>no downtime</strong></li>
</ul>

<h3 id="102-rollout-history">10.2 Rollout History</h3>
<p>Because Deployments keep multiple ReplicaSets, Kubernetes also keeps a history of changes.</p>

<p>This allows us to:</p>
<ul>
  <li>See previous versions</li>
  <li>Roll back if an update fails</li>
</ul>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl rollout <span class="nb">history </span>deployment nginx-deployment

<span class="c"># Example Output</span>
REVISION  CHANGE-CAUSE
1         Initial deployment
2         Updated nginx image
</code></pre></div></div>

<h3 id="103-rollback-to-a-previous-version">10.3 Rollback to a Previous Version</h3>
<p>If something breaks after an update, we can return to the previous version:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl rollout undo deployment nginx-deployment
</code></pre></div></div>

<p>This command restores the <strong>previous ReplicaSet and its Pods</strong>.</p>

<h2 id="11-node-selector">11. Node Selector</h2>
<p>A single worker nodes might not have enough resources (CPU, memory) to handle the load of all the pods.</p>

<p>So it is recommended to have multiple worker nodes. Also, <strong>it is not necessary for all worker nodes to have the same hardware specification</strong>.</p>

<p>We can check nodes by running <code class="language-plaintext highlighter-rouge">kubectl get nodes</code> command.</p>

<p>With multiple nodes, we can use <strong>Node Selector</strong> to control where Pods are scheduled.</p>

<p>A <strong>Node Selector</strong> is a simple way to control which Node a Pod should run on.</p>

<p>It works by using labels on Nodes. When a Pod has a node selector, Kubernetes will schedule the Pod only on Nodes that have the <strong>matching label</strong>.</p>

<blockquote>
  <p>Node Selector tells Kubernetes to run a Pod on a specific group of Nodes.</p>
</blockquote>

<p>So, how it works:</p>

<ol>
  <li>First, we label our Nodes with specific key-value pairs.</li>
</ol>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl label nodes worker-node-1 <span class="nb">env</span><span class="o">=</span>production
kubectl label nodes worker-node-2 <span class="nb">env</span><span class="o">=</span>production
kubectl label nodes worker-node-3 <span class="nb">env</span><span class="o">=</span>staging
</code></pre></div></div>
<ol>
  <li>Second, we add a Node Selector to our Pod definition to specify that it should only run on Nodes with the label <code class="language-plaintext highlighter-rouge">env=production</code>.</li>
</ol>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">nginx-pod</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">nodeSelector</span><span class="pi">:</span>
    <span class="na">env</span><span class="pi">:</span> <span class="s">production</span>
  <span class="na">containers</span><span class="pi">:</span>
    <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
      <span class="na">image</span><span class="pi">:</span> <span class="s">nginx</span>
</code></pre></div></div>

<p>In this example, the Pod will only be scheduled on Nodes that have the label <code class="language-plaintext highlighter-rouge">env=production</code>.</p>

<p>If no Node has that label, the Pod will stay in <strong>Pending state</strong>.</p>

<p><strong>Node selectors</strong> are useful when we want to run Pods on specific hardware or environments, such as:</p>
<ul>
  <li>Nodes with <strong>SSD storage</strong></li>
  <li>Nodes with <strong>GPU</strong></li>
  <li>Nodes for <strong>production workloads</strong></li>
  <li>Nodes in a specific <strong>region or zone</strong></li>
</ul>

<h2 id="12-node-affinity">12. Node Affinity</h2>
<p><strong>Node Affinity</strong> is a Kubernetes feature that controls which Nodes a Pod can run on, based on Node labels.</p>

<p>It is similar to <strong>Node Selector</strong>, but it is more flexible.</p>

<blockquote>
  <p>Node Affinity allows Pods to be scheduled on Nodes that match specific label rules.</p>
</blockquote>

<p>While Node Selector only supports simple matching, <strong>Node Affinity</strong> allows:</p>
<ul>
  <li>More complex rules</li>
  <li>Multiple conditions</li>
  <li>Preferred scheduling</li>
</ul>

<p>Because of this, Node Affinity is often used instead of Node Selector in more advanced setups.</p>

<h3 id="121-types-of-node-affinity">12.1 Types of Node Affinity</h3>
<p>There are two main types.</p>

<h4 id="1211-required-node-affinity">12.1.1 Required Node Affinity</h4>
<p>This is a <strong>strict rule</strong>.</p>

<p>The Pod must run on a Node that matches the condition.</p>

<p>If no matching Node exists, the Pod will stay in <strong>Pending</strong>.</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">affinity</span><span class="pi">:</span>
    <span class="na">nodeAffinity</span><span class="pi">:</span>
      <span class="na">requiredDuringSchedulingIgnoredDuringExecution</span><span class="pi">:</span>
        <span class="na">nodeSelectorTerms</span><span class="pi">:</span>
        <span class="pi">-</span> <span class="na">matchExpressions</span><span class="pi">:</span>
          <span class="pi">-</span> <span class="na">key</span><span class="pi">:</span> <span class="s">Size</span>
            <span class="na">operator</span><span class="pi">:</span> <span class="s">In</span>
            <span class="na">values</span><span class="pi">:</span>
            <span class="pi">-</span> <span class="s">m5.2xlarge</span>
  <span class="na">containers</span><span class="pi">:</span>
  <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s">nginx</span>
</code></pre></div></div>

<p>In this example, the Pod will only be scheduled on Nodes with the label <code class="language-plaintext highlighter-rouge">Size = m5.2xlarge</code>.</p>

<h4 id="1211-preferred-node-affinity">12.1.1 Preferred Node Affinity</h4>
<p>This is a <strong>soft rule</strong>.</p>

<p>Kubernetes will <strong>try to schedule the Pod on matching Nodes</strong>, but if none exist, it will still run the Pod on other Nodes.</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">affinity</span><span class="pi">:</span>
    <span class="na">nodeAffinity</span><span class="pi">:</span>
      <span class="na">preferredDuringSchedulingIgnoredDuringExecution</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="na">weight</span><span class="pi">:</span> <span class="m">1</span>
        <span class="na">preference</span><span class="pi">:</span>
          <span class="na">matchExpressions</span><span class="pi">:</span>
          <span class="pi">-</span> <span class="na">key</span><span class="pi">:</span> <span class="s">Size</span>
            <span class="na">operator</span><span class="pi">:</span> <span class="s">In</span>
            <span class="na">values</span><span class="pi">:</span>
            <span class="pi">-</span> <span class="s">m5.2xlarge</span>
  <span class="na">containers</span><span class="pi">:</span>
  <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s">nginx</span>
</code></pre></div></div>

<p>In this example, Kubernetes will prefer to schedule the Pod on Nodes with <code class="language-plaintext highlighter-rouge">Size = m5.2xlarge</code>, but if none are available, it will still schedule the Pod on other Nodes.</p>

<h3 id="122-common-operators">12.2 Common Operators</h3>
<ul>
  <li><code class="language-plaintext highlighter-rouge">In</code>: Label value must match</li>
  <li><code class="language-plaintext highlighter-rouge">NotIn</code>: Label value must not match</li>
  <li><code class="language-plaintext highlighter-rouge">Exists</code>: Label key must exist</li>
  <li><code class="language-plaintext highlighter-rouge">DoesNotExist</code>: Label key must not exist</li>
</ul>

<h2 id="13-daemonset">13. Daemonset</h2>
<p>A DaemonSet is a Kubernetes resource that ensures <strong>a Pod runs on every Node in the cluster</strong>.</p>

<p>This means Kubernetes will automatically create <strong>one Pod per Node</strong>.</p>

<p>If a <strong>new Node joins the cluster</strong>, the DaemonSet will automatically create a Pod on that Node as well.</p>

<blockquote>
  <p>A DaemonSet makes sure a specific Pod runs on all (or selected) Nodes.</p>
</blockquote>

<p><strong>How DaemonSet Works:</strong></p>
<ul>
  <li>Kubernetes checks all available Nodes.</li>
  <li>It schedules one Pod on each Node.</li>
  <li>If a new Node is added, a new Pod is automatically created.</li>
  <li>If a Node is removed, the Pod on that Node is removed as well.</li>
</ul>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Node 1 → Pod
Node 2 → Pod
Node 3 → Pod

</code></pre></div></div>

<p>DaemonSets are typically used for system-level services that need to run on every Node.</p>

<p>Examples:</p>
<ul>
  <li><strong>Log collection agents</strong> (collect logs from every Node)</li>
  <li><strong>Monitoring agents</strong></li>
  <li><strong>Network plugins</strong></li>
  <li><strong>Security monitoring tools</strong></li>
</ul>

<p>These services must run on <strong>all Nodes</strong>, which makes DaemonSet the perfect resource.</p>

<p>Example DaemonSet Manifest:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">apps/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">DaemonSet</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">log-agent</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">selector</span><span class="pi">:</span>
    <span class="na">matchLabels</span><span class="pi">:</span>
      <span class="na">app</span><span class="pi">:</span> <span class="s">log-agent</span>
  <span class="na">template</span><span class="pi">:</span>
    <span class="na">metadata</span><span class="pi">:</span>
      <span class="na">labels</span><span class="pi">:</span>
        <span class="na">app</span><span class="pi">:</span> <span class="s">log-agent</span>
    <span class="na">spec</span><span class="pi">:</span>
      <span class="na">containers</span><span class="pi">:</span>
        <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">log-agent</span>
          <span class="na">image</span><span class="pi">:</span> <span class="s">fluentd</span>
          <span class="na">resources</span><span class="pi">:</span>
            <span class="na">limits</span><span class="pi">:</span>
              <span class="na">memory</span><span class="pi">:</span> <span class="s2">"</span><span class="s">200Mi"</span>
              <span class="na">cpu</span><span class="pi">:</span> <span class="s2">"</span><span class="s">500m"</span>
</code></pre></div></div>

<p>In this example, a Pod running the <code class="language-plaintext highlighter-rouge">fluentd</code> log agent will be created on every Node in the cluster.</p>

<h2 id="requests-and-limits">Requests and Limits</h2>
<p>In Kubernetes, Requests and Limits are used to control how much CPU and memory a container can use.</p>

<p>They help Kubernetes schedule Pods properly and prevent one container from using too many resources.</p>

<blockquote>
  <p>Requests define the minimum resources a container needs, while Limits define the maximum resources it can use.</p>
</blockquote>

<h3 id="requests">Requests</h3>
<p>A request is the amount of CPU or memory that a container needs to run.</p>

<p>Kubernetes uses this value to decide which Node is suitable for the Pod.</p>

<blockquote>
  <p>If a container requests 500Mi memory, Kubernetes will place it on a Node that has at least that much available memory.</p>
</blockquote>

<h3 id="limits">Limits</h3>
<p>A limit is the maximum amount of CPU or memory a container is allowed to use.</p>

<p>If the container tries to use more than the limit:</p>
<ul>
  <li><strong>CPU</strong> → it will be throttled (slowed down)</li>
  <li><strong>Memory</strong> → the container may be terminated (OOMKilled)</li>
</ul>

<p>Example of Requests and Limits:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">nginx-pod</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">containers</span><span class="pi">:</span>
    <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
      <span class="na">image</span><span class="pi">:</span> <span class="s">nginx</span>
      <span class="na">resources</span><span class="pi">:</span>
        <span class="na">requests</span><span class="pi">:</span>
          <span class="na">memory</span><span class="pi">:</span> <span class="s2">"</span><span class="s">128Mi"</span>
          <span class="na">cpu</span><span class="pi">:</span> <span class="s2">"</span><span class="s">250m"</span>
        <span class="na">limits</span><span class="pi">:</span>
          <span class="na">memory</span><span class="pi">:</span> <span class="s2">"</span><span class="s">256Mi"</span>
          <span class="na">cpu</span><span class="pi">:</span> <span class="s2">"</span><span class="s">500m"</span>
</code></pre></div></div>

<p>In this example:</p>
<ul>
  <li>The container requests 128Mi of memory and 250m of CPU.</li>
  <li>The container can use up to 256Mi of memory and 500m of CPU. This is the maximum.</li>
</ul>

<p>Requests and Limits help us to:</p>
<ul>
  <li>Prevent resource starvation</li>
  <li>Improve cluster stability</li>
  <li>Ensure fair resource usage</li>
  <li>Help Kubernetes schedule Pods efficiently</li>
</ul>

<h2 id="priority-class">Priority Class</h2>
<p>PriorityClass is a Kubernetes resource used to define the priority level of Pods.</p>

<p>It tells Kubernetes which Pods are more important when scheduling or when resources are limited.</p>

<blockquote>
  <p>PriorityClass helps Kubernetes decide which Pods should run first.</p>
</blockquote>

<p><strong>Why PriorityClass is Useful</strong></p>

<p>In a cluster with limited resources, not all Pods can run at the same time.</p>

<p>PriorityClass helps Kubernetes decide which Pods should be scheduled first.</p>

<p>If the cluster does not have enough resources, Kubernetes may evict lower-priority Pods to make room for higher-priority Pods.</p>

<p>This process is called <strong>Pod preemption</strong>.</p>

<p><strong>How PriorityClass Works</strong></p>
<ul>
  <li>First, we create a PriorityClass with a priority value.</li>
  <li>Pods reference that PriorityClass.</li>
  <li>Kubernetes uses the priority value to decide scheduling order.</li>
</ul>

<blockquote>
  <p><strong>Higher value</strong> = higher priority.</p>
</blockquote>

<p>Example of PriorityClass:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">scheduling.k8s.io/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">PriorityClass</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">high-priority</span>
<span class="na">value</span><span class="pi">:</span> <span class="m">1000</span>
<span class="na">globalDefault</span><span class="pi">:</span> <span class="no">false</span>
<span class="na">description</span><span class="pi">:</span> <span class="s2">"</span><span class="s">This</span><span class="nv"> </span><span class="s">priority</span><span class="nv"> </span><span class="s">class</span><span class="nv"> </span><span class="s">is</span><span class="nv"> </span><span class="s">for</span><span class="nv"> </span><span class="s">critical</span><span class="nv"> </span><span class="s">applications"</span>
</code></pre></div></div>

<p>In this example:</p>
<ul>
  <li>We create a PriorityClass named <code class="language-plaintext highlighter-rouge">high-priority</code> with a value of <code class="language-plaintext highlighter-rouge">1000</code>.</li>
  <li>Pods that reference this PriorityClass will have a high priority when scheduling.</li>
</ul>

<p>Example of a Pod referencing PriorityClass:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">critical-app</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">priorityClassName</span><span class="pi">:</span> <span class="s">high-priority</span>
  <span class="na">containers</span><span class="pi">:</span>
    <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">app</span>
      <span class="na">image</span><span class="pi">:</span> <span class="s">critical-app-image</span>
</code></pre></div></div>
<p>In this example, the <code class="language-plaintext highlighter-rouge">critical-app</code> Pod will have a high priority when Kubernetes schedules it.</p>

<p><strong>Another example scenario</strong></p>

<p>Imagine the cluster is full:</p>
<ul>
  <li>Pod A → priority 1000</li>
  <li>Pod B → priority 100</li>
</ul>

<p>If resources run out, Kubernetes may stop <code class="language-plaintext highlighter-rouge">Pod B</code> so <code class="language-plaintext highlighter-rouge">Pod A</code> can run.</p>

<h2 id="overview-of-service">Overview of Service</h2>
<p>In Kubernetes, every <strong>Pod</strong> gets its own <strong>private IP address</strong>.</p>

<p>Pods inside the same cluster can communicate using these private IPs.</p>

<h3 id="use-case-front-end-and-back-end">Use Case: Front End and Back End</h3>

<p>Let’s say we have:</p>
<ul>
  <li><strong>A frontend Pod</strong></li>
  <li><strong>A backend Pod</strong></li>
</ul>

<p>The frontend needs to call the backend.</p>

<p>Well, we can refer to the backend pod IPs. But, this will creates several problems:</p>
<ul>
  <li>Pods are <strong>ephemeral</strong> (they can be deleted and recreated)</li>
  <li>When recreated, they <strong>get a new IP address</strong></li>
  <li>If we hardcoded the IP, our app will be break</li>
</ul>

<p>If the backend pods are running as deployment, there is problems like:</p>
<ul>
  <li>We will have multiple backend Pods</li>
  <li>Each Pod has a different IP</li>
  <li>Hardcoding all IPs is not practical</li>
</ul>

<p>Also, we want to:</p>

<p>Distribute traffic across all backend Pods (load balancing)</p>

<p>This is where <strong>Service</strong> comes in.</p>

<h3 id="introduction-of-services">Introduction of Services</h3>
<p>A Service provides a <strong>stable IP and DNS name</strong> to access a group of Pods.</p>

<p>Instead of calling Pod IPs, the frontend calls the <strong>Service</strong>.</p>

<p><strong>So, how it works?</strong></p>
<ul>
  <li>We create a Service.</li>
  <li>The Service uses a selector to find matching Pods.</li>
  <li>Kubernetes automatically creates Endpoints (list of Pod IPs).</li>
  <li>Traffic is distributed across those Pods.</li>
</ul>

<p><strong>Endpoints</strong> are the actual Pod IP addresses behind a Service.</p>
<ul>
  <li><strong>Service</strong> = stable entry point</li>
  <li><strong>Endpoints</strong> = real Pods</li>
</ul>

<p>So the flow is like:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Frontend → Service → Endpoints <span class="o">(</span>Backend Pods<span class="o">)</span>
</code></pre></div></div>

<ul>
  <li>Frontend calls Service</li>
  <li>Service forwards request to one of the backend Pods</li>
  <li>Load is distributed automatically</li>
</ul>

<h2 id="practical-service-and-endpoints">Practical Service and Endpoints</h2>

<p>Let’s see a <strong>simple real-world example</strong> of how Service and Endpoints work together.</p>

<h3 id="step-1-backend-deployment">Step 1: Backend Deployment</h3>

<p>First, we create backend Pods using a Deployment:</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">apps/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Deployment</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">backend-deployment</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">replicas</span><span class="pi">:</span> <span class="m">3</span>
  <span class="na">selector</span><span class="pi">:</span>
    <span class="na">matchLabels</span><span class="pi">:</span>
      <span class="na">app</span><span class="pi">:</span> <span class="s">backend</span>
  <span class="na">template</span><span class="pi">:</span>
    <span class="na">metadata</span><span class="pi">:</span>
      <span class="na">labels</span><span class="pi">:</span>
        <span class="na">app</span><span class="pi">:</span> <span class="s">backend</span>
    <span class="na">spec</span><span class="pi">:</span>
      <span class="na">containers</span><span class="pi">:</span>
        <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">backend</span>
          <span class="na">image</span><span class="pi">:</span> <span class="s">nginx</span>
          <span class="na">ports</span><span class="pi">:</span>
            <span class="pi">-</span> <span class="na">containerPort</span><span class="pi">:</span> <span class="m">80</span>
</code></pre></div></div>

<p>This creates 3 backend Pods with the label <code class="language-plaintext highlighter-rouge">app=backend</code>. Each pod has it’s own IP.</p>

<h3 id="step-2-create-a-service">Step 2: Create a Service</h3>

<p>Now, create a Service to expose these Pods:</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Service</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">backend-service</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">selector</span><span class="pi">:</span>
    <span class="na">app</span><span class="pi">:</span> <span class="s">backend</span>
  <span class="na">ports</span><span class="pi">:</span>
    <span class="pi">-</span> <span class="na">port</span><span class="pi">:</span> <span class="m">80</span>
      <span class="na">targetPort</span><span class="pi">:</span> <span class="m">80</span>
</code></pre></div></div>
<p>This Service will:</p>
<ul>
  <li>Select Pods with <code class="language-plaintext highlighter-rouge">app=backend</code></li>
  <li>Expose port 80</li>
</ul>

<h3 id="step-3-what-happens-internally">Step 3: What Happens Internally</h3>

<p>When the Service is created:</p>
<ol>
  <li>Kubernetes looks at the selector:</li>
</ol>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">app</span><span class="pi">:</span> <span class="s">backend</span>
</code></pre></div></div>

<ol>
  <li>It finds all matching Pods</li>
  <li>It automatically creates Endpoints</li>
</ol>

<h3 id="step-4-endpoints-example">Step 4: Endpoints Example</h3>

<p>We can check endpoints with:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl get endpoints backend-service

<span class="c"># Example Output</span>
NAME              ENDPOINTS
backend-service   10.1.1.2:80, 10.1.1.3:80, 10.1.1.4:80
</code></pre></div></div>

<p>These IP are actual backend Pods.</p>

<h3 id="step-5-traffic-flow">Step 5: Traffic Flow</h3>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Frontend Pod → backend-service → Backend Pods
</code></pre></div></div>

<ul>
  <li>Frontend calls <code class="language-plaintext highlighter-rouge">backend-service</code></li>
  <li>Service forwards request to one of the backend Pods</li>
  <li>Load is balanced across all backend Pods</li>
</ul>

<p>So, even if:</p>
<ul>
  <li>A Pod crashes</li>
  <li>A new Pod is created</li>
  <li>IPs change</li>
</ul>

<p>Kubernetes will:</p>
<ul>
  <li>Update Endpoints automatically</li>
  <li>Keep the Service working</li>
</ul>

<h2 id="types-of-services">Types of Services</h2>

<p>Kubernetes provides different types of Services to expose applications based on how they should be accessed.</p>

<h3 id="1-clusterip-default">1. ClusterIP (Default)</h3>

<p>ClusterIP is the default Service type.</p>

<p>It exposes the application only inside the Kubernetes cluster.</p>

<p>Pods can communicate with the Service using its DNS name or Cluster IP, but users outside the cluster cannot access it directly.</p>

<h4 id="use-cases">Use Cases</h4>
<ul>
  <li>Frontend communicating with backend APIs</li>
  <li>Communication between microservices</li>
  <li>Internal databases</li>
  <li>Internal caching services such as Redis</li>
</ul>

<p>Example</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Frontend Pod → backend-service → Backend Pods
</code></pre></div></div>

<h3 id="2-nodeport">2. NodePort</h3>
<p>NodePort exposes the Service on a specific port of every Node in the cluster.</p>

<p>The application can be accessed using:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>http://&lt;NodeIP&gt;:&lt;NodePort&gt;
</code></pre></div></div>

<p>Kubernetes forwards traffic from the NodePort to the Service and then to the Pods.</p>

<p>Use Cases:</p>
<ul>
  <li>Development and testing environments</li>
  <li>Learning Kubernetes concepts</li>
  <li>Small on-premise deployments</li>
  <li>Situations where a cloud load balancer is not available</li>
</ul>

<p>Example:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>User → Node IP:30080 → Service → Pods
</code></pre></div></div>

<h3 id="3-loadbalancer">3. LoadBalancer</h3>

<p>LoadBalancer exposes the Service through an external load balancer provided by the cloud platform such as AWS, Google Cloud, and Azure.</p>

<p>When the Service is created, Kubernetes requests a public IP from the cloud provider.</p>

<p>Use Cases:</p>
<ul>
  <li>Public websites</li>
  <li>Public APIs</li>
  <li>Production applications</li>
  <li>Applications that need internet access</li>
</ul>

<p>Example</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Internet → LoadBalancer → Service → Pods
</code></pre></div></div>

<h3 id="4-externalname">4. ExternalName</h3>

<p>ExternalName maps a Kubernetes Service to an external DNS name.</p>

<p>Instead of forwarding traffic to Pods, Kubernetes returns the external DNS name.</p>

<p>Use Cases:</p>
<ul>
  <li>Connecting to external databases</li>
  <li>Accessing third-party APIs</li>
  <li>Migrating applications without changing code</li>
</ul>

<p>Example</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Application → external-service → External DNS <span class="o">(</span>e.g., api.example.com<span class="o">)</span>
</code></pre></div></div>

<h2 id="ingress">Ingress</h2>
<p>As applications grow, a cluster may contain many Services such as:</p>
<ul>
  <li>Frontend Service</li>
  <li>Backend API Service</li>
  <li>Authentication Service</li>
  <li>Admin Service</li>
</ul>

<p>If each Service is exposed using a LoadBalancer, we’ll end up with multiple external IP addresses, which can be expensive and difficult to manage.</p>

<p>This is where Ingress comes in.</p>

<p>Ingress provides a single entry point to route external traffic to different Services inside the cluster.</p>

<h3 id="ingress-resource">Ingress Resource</h3>

<p>An Ingress is a Kubernetes resource that defines routing rules for incoming HTTP and HTTPS traffic.</p>

<p>It tells Kubernetes:</p>
<ul>
  <li>Which domain should be used</li>
  <li>Which path should be matched</li>
  <li>Which Service should receive the request</li>
</ul>

<p>Example:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>app.example.com      → frontend-service
api.example.com      → backend-service
admin.example.com    → admin-service

<span class="c"># Using path</span>
example.com/         → frontend-service
example.com/api      → backend-service
example.com/admin    → admin-service
</code></pre></div></div>

<p>Example of Ingress Resource:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">networking.k8s.io/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Ingress</span>
<span class="na">metadata</span><span class="pi">:</span>
  <span class="na">name</span><span class="pi">:</span> <span class="s">my-ingress</span>
<span class="na">spec</span><span class="pi">:</span>
  <span class="na">rules</span><span class="pi">:</span>
    <span class="pi">-</span> <span class="na">host</span><span class="pi">:</span> <span class="s">example.com</span>
      <span class="na">http</span><span class="pi">:</span>
        <span class="na">paths</span><span class="pi">:</span>
          <span class="pi">-</span> <span class="na">path</span><span class="pi">:</span> <span class="s">/api</span>
            <span class="na">pathType</span><span class="pi">:</span> <span class="s">Prefix</span>
            <span class="na">backend</span><span class="pi">:</span>
              <span class="na">service</span><span class="pi">:</span>
                <span class="na">name</span><span class="pi">:</span> <span class="s">backend-service</span>
                <span class="na">port</span><span class="pi">:</span>
                  <span class="na">number</span><span class="pi">:</span> <span class="m">80</span>
</code></pre></div></div>

<p>This rule means <code class="language-plaintext highlighter-rouge">example.com/api → backend-service</code>.</p>

<h2 id="overview-of-helm">Overview of Helm</h2>
<p>As Kubernetes applications become more complex, managing YAML files can become difficult.</p>

<p>A typical application may require multiple resources:</p>

<ul>
  <li>Deployment</li>
  <li>Service</li>
  <li>ConfigMap</li>
  <li>Secret</li>
  <li>Ingress</li>
  <li>PersistentVolumeClaim</li>
</ul>

<p>Managing all these YAML files manually can be time-consuming and error-prone.</p>

<p>This is where Helm helps.</p>

<p>Helm is a package manager for Kubernetes.</p>

<h2 id="namespace">Namespace</h2>
<p>A Namespace is a way to logically divide and organize resources within a Kubernetes cluster.</p>

<p><strong>Why Do We Need Namespaces?</strong></p>

<p>Let’s say a cluster used by multiple teams:</p>
<ul>
  <li>Development team</li>
  <li>Testing team</li>
  <li>Production team</li>
</ul>

<p>Without namespaces, all resources would exist together, making them difficult to manage and potentially causing naming conflicts.</p>

<p>Namespaces solve this problem.</p>

<p>Example:</p>

<p>Instead of one shared space:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Cluster
├── frontend-deployment
├── backend-deployment
└── database
</code></pre></div></div>

<p>We can create separate resources:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Cluster
├── dev
│   ├── frontend-deployment
│   └── backend-deployment
│
├── <span class="nb">test</span>
│   ├── frontend-deployment
│   └── backend-deployment
│
└── prod
    ├── frontend-deployment
    └── backend-deployment
</code></pre></div></div>

<p>Each team can have its own namespace, allowing them to manage their resources independently.</p>

<h2 id="service-account">Service Account</h2>
<p>Kubernetes Cluster have two categories of account:</p>
<ul>
  <li>User Accounts (For Humans)</li>
  <li>Service Accounts (For Application)</li>
</ul>

<p>Service Account is an identity used by applications and Pods to interact with the Kubernetes API.</p>

<p>Sometimes an application needs to interact with Kubernetes.</p>

<p>For example, an application may need to:</p>
<ul>
  <li>Read Pods</li>
  <li>Create Jobs</li>
  <li>Watch Deployments</li>
  <li>Access ConfigMaps</li>
  <li>Query cluster information</li>
</ul>

<p>To do this securely, the Pod needs an identity. That’s where a Service Account comes in.</p>

<p>When a Pod uses a Service Account:</p>
<ol>
  <li>Kubernetes provides credentials to the Pod.</li>
  <li>The Pod uses those credentials to authenticate.</li>
  <li>Kubernetes checks what permissions the Service Account has.</li>
  <li>The action is allowed or denied.</li>
</ol>

<p>FLow:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Pod
  ↓
Service Account
  ↓
Kubernetes API
  ↓
Permission Check <span class="o">(</span>RBAC<span class="o">)</span>
</code></pre></div></div>

<h3 id="default-service-account">Default Service Account</h3>
<p>Every Namespace automatically has a Service Account called <code class="language-plaintext highlighter-rouge">default</code>.</p>

<p>If we don’t specify a Service Account, our Pod will use the default one.</p>

<p>To view service account:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl get serviceaccounts
</code></pre></div></div>]]></content><author><name>Nugroho</name></author><category term="kubernetes" /><category term="aws" /><category term="k8s" /><category term="containers" /><summary type="html"><![CDATA[it’s just kubernetes notes.]]></summary></entry><entry><title type="html">Terraform Notes</title><link href="https://galnug.com/terraform-notes/" rel="alternate" type="text/html" title="Terraform Notes" /><published>2026-01-14T21:00:00+07:00</published><updated>2026-01-14T21:00:00+07:00</updated><id>https://galnug.com/terraform-notes</id><content type="html" xml:base="https://galnug.com/terraform-notes/"><![CDATA[<p>it’s just terraform notes.</p>

<h2 id="1-the-background">1. The Background</h2>

<p>There are two ways to manage infrastructure:</p>

<ol>
  <li>Manually</li>
  <li>Through automation</li>
</ol>

<p>When I first learned AWS, I created everything using the console. I clicked here and there to create EC2, S3, VPC, and other resources. At that time, it felt easy and normal. For learning, I think using the console helped me understand what each service does.</p>

<p>The problem started when my project became bigger. I had many resources. Some of them depended on each other. For example, an EC2 instance needs a VPC, subnet, security group, and IAM role. If I deleted one thing by mistake, other resources stopped working. Sometimes I even forgot what I created before.</p>

<p>Another problem was when I wanted to create the same setup again. I had to repeat all the steps manually. It took time, and sometimes I missed some settings. The result was not always the same.</p>

<p>So this manual approach can be solved by using Infrastructure as Code (IaC), such as Terraform or CloudFormation. Instead of creating resources one by one from the console, we define everything in code.</p>

<p>In this article, I will write notes about my journey learning Terraform. I will not explain the concepts in very detailed way because this is just for my personal note. I write this so I can come back and read it again if I forget some concepts in the future.</p>

<h2 id="2-iac-categories">2. IAC Categories</h2>
<h3 id="21-infrastrucure-orchestration">2.1 Infrastrucure Orchestration</h3>
<p>Infrastructure Orchestration is used to create and manage infrastructure. It focuses on provisioning resources like servers, networks, and databases.</p>

<p>Example:</p>
<ul>
  <li>Create 3 EC2 instances with 8 GB RAM and 4 vCPUs</li>
  <li>Allow SSH access only from my IP address</li>
  <li>Create VPC, subnet, and security group</li>
</ul>

<p>Tools example: Terraform, CloudFormation</p>

<h3 id="22-configuration-management">2.2 Configuration Management</h3>
<p>Configuration Management is used to manage software inside the server. It focuses on installing and configuring applications after the server is created.</p>

<p>Example:</p>
<ul>
  <li>Install Nginx on EC2</li>
  <li>Configure Nginx to serve a static website</li>
  <li>Install Antivirus version 11.0.0 on all EC2 instances</li>
</ul>

<p>Tools example: Ansible, Chef, Puppet</p>

<h2 id="3-authentication-and-authorization">3. Authentication and Authorization</h2>

<ol>
  <li>Authentication is the process of verifying who a user is.</li>
  <li>Authorization is the process of verifying what they are allowed to access.</li>
</ol>

<p>When using Terraform, make sure you have access to AWS (or another cloud provider) and you have permission to create resources. If you only have ReadOnly permission, you cannot create or modify resources.
To connect Terraform with AWS, you need credentials. You can use:</p>
<ol>
  <li>Temporary access key (recommended)</li>
  <li>IAM User access key</li>
  <li>Github Tokens</li>
</ol>

<p>You can visit these documentation to configure credentials using <a href="https://docs.aws.amazon.com/cli/latest/userguide/getting-started-quickstart.html">AWS CLI</a></p>

<h2 id="4-providers-and-resources">4. Providers and Resources</h2>
<p>Terraform supports multiple providers such as AWS, Azure, Google Cloud, Kubernetes, Oracle Cloud, and Alibaba Cloud.</p>

<h3 id="41-provider-plugins">4.1 Provider Plugins</h3>

<p>When you run terraform init, Terraform will download the provider plugin automatically. The plugin will be saved in the .terraform directory.</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">terraform</span> <span class="p">{</span>
  <span class="nx">required_providers</span> <span class="p">{</span>
    <span class="nx">aws</span> <span class="p">=</span> <span class="p">{</span>
      <span class="nx">source</span> <span class="p">=</span> <span class="s2">"hashicorp/aws"</span>
      <span class="c1"># Constrain to a specific version range</span>
      <span class="nx">version</span> <span class="p">=</span> <span class="s2">"~&gt; 6.7.0"</span> 
    <span class="p">}</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># AWS Provider with a specific region</span>
<span class="nx">provider</span> <span class="s2">"aws"</span> <span class="p">{</span>
  <span class="nx">region</span> <span class="p">=</span> <span class="s2">"us-east-1"</span>
<span class="p">}</span>
</code></pre></div></div>

<p>For production use, you should always used <code class="language-plaintext highlighter-rouge">version</code> argument to ensure new version doesn’t break the current version.</p>

<h3 id="42-resources">4.2 Resources</h3>
<p>A resource block is used to create infrastructure.</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web_server1"</span> <span class="p">{</span>
  <span class="c1"># configuration here</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Explanation:</p>
<ul>
  <li><code class="language-plaintext highlighter-rouge">aws_instance</code> → resource type</li>
  <li><code class="language-plaintext highlighter-rouge">web_server1</code> → local name (identifier)</li>
</ul>

<p>The local name must be unique in the same project.</p>

<h2 id="5-terraform-destroy">5. Terraform Destroy</h2>
<p>Use the following command to destroy all resources created by Terraform.</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Destroy all resources</span>
<span class="nx">terraform</span> <span class="nx">destroy</span>

<span class="c1"># Destroy specific resources</span>
<span class="nx">terraform</span> <span class="nx">destroy</span> <span class="err">-</span><span class="nx">target</span><span class="err">=&lt;</span><span class="nx">resource_type</span><span class="err">&gt;.&lt;</span><span class="nx">resource_name</span><span class="err">&gt;</span>

<span class="c1"># Example</span>
<span class="nx">terraform</span> <span class="nx">destroy</span> <span class="err">-</span><span class="nx">target</span><span class="err">=</span><span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web_server1</span>
</code></pre></div></div>

<h2 id="6-terraform-state-file">6. Terraform State File</h2>
<p>Terraform state file is a file that stores information about the resources Terraform manages.
The filename is <code class="language-plaintext highlighter-rouge">terraform.tfstate</code>.</p>

<p>Terraform uses this file to:</p>
<ul>
  <li>Track what resources are created</li>
  <li>Know the current state of infrastructure</li>
  <li>Compare real infrastructure with your code</li>
</ul>

<h2 id="7-desired-and-current-state">7. Desired and Current State</h2>
<h3 id="71-desired-state">7.1 Desired State</h3>
<p>Desired state is what you define in your <code class="language-plaintext highlighter-rouge">.tf</code> files. It is the condition you want your infrastructure to be.</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web_server1"</span> <span class="p">{</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Here, your desired state is: You want one EC2 instance with type <code class="language-plaintext highlighter-rouge">t3.micro</code>.</p>

<h3 id="72-current-state">7.2 Current State</h3>
<p>Current state is the real condition of your infrastructure right now. Terraform gets this information from:</p>
<ol>
  <li>The terraform.tfstate file</li>
  <li>The real infrastructure in AWS</li>
</ol>

<h4 id="721-how-it-works">7.2.1 How it works?</h4>
<ol>
  <li>When you run <code class="language-plaintext highlighter-rouge">terraform plan</code>, Terraform will check the current state of your infrastructure.</li>
  <li>Terraform compares:
    <ul>
      <li>Desired state (your code)</li>
      <li>Current state (real infrastructure)</li>
    </ul>
  </li>
  <li>If they are different, Terraform will show what needs to change. (TF Drift)</li>
  <li>When you run <code class="language-plaintext highlighter-rouge">terraform apply</code> Terraform will make the current state match the desired state.</li>
</ol>

<p class="notice--danger"><strong>Notes!</strong> Always make sure the current state and desired state are the same. The configuration in AWS Console must match your Terraform script. If they are different and you run <code class="language-plaintext highlighter-rouge">terraform apply</code>, Terraform will detect the changes and modify the resources to match your code.</p>

<p>So, in summary:</p>
<ul>
  <li>Desired state → What you want (tf scripts)</li>
  <li>Current state → What exists now (in AWS Console)</li>
  <li>Terraform → Makes them the same (<code class="language-plaintext highlighter-rouge">terraform apply</code>)</li>
</ul>

<p>I always fix <code class="language-plaintext highlighter-rouge">tf drift</code> when this is happen.</p>

<h2 id="8-terraform-refresh">8. Terraform Refresh</h2>
<p><code class="language-plaintext highlighter-rouge">terraform refresh</code> is used to update the Terraform state file with the real condition of your infrastructure.
It reads all resources from the cloud provider and updates the <code class="language-plaintext highlighter-rouge">terraform.tfstate</code> file.</p>

<p>Important:</p>
<ul>
  <li>It does NOT change real resources in AWS.</li>
  <li>It only updates the Terraform state file.
Sometimes, resources are changed manually in AWS console.</li>
</ul>

<p>Example:</p>

<p>You change EC2 instance type from <code class="language-plaintext highlighter-rouge">t3.micro</code> to <code class="language-plaintext highlighter-rouge">t3.small</code> directly in AWS Console.</p>

<p>Now:</p>
<ol>
  <li>Real infrastructure → <code class="language-plaintext highlighter-rouge">t3.small</code></li>
  <li>Terraform state → still <code class="language-plaintext highlighter-rouge">t3.micro</code></li>
</ol>

<p>The state is not accurate anymore. Running <a href="https://developer.hashicorp.com/terraform/cli/commands/refresh"><code class="language-plaintext highlighter-rouge">terraform refresh</code></a> will sync the state with the real infrastructure.</p>

<p>It is equal to:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform apply <span class="nt">-refresh-only</span> <span class="nt">-auto-approve</span>
</code></pre></div></div>

<p>Recommended way:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform apply <span class="nt">-refresh-only</span>
</code></pre></div></div>

<p>This will:</p>
<ol>
  <li>Check real infrastructure</li>
  <li>Show detected changes</li>
  <li>Ask for confirmation before updating state</li>
</ol>

<p>Simple idea:</p>
<ul>
  <li>Refresh → Sync state with real infrastructure</li>
  <li>Does not create or delete resources</li>
  <li>Only updates terraform.tfstate and it doesn’t update tf scripts.</li>
</ul>

<h2 id="9-aws---authentication-and-configurations">9. AWS - Authentication and Configurations</h2>
<p>I don’t want to explain too much in this section because the concept is the same as when using AWS CLI.</p>

<p>Before using Terraform with AWS, make sure to check which account you are currently using, run:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>aws sts get-caller-identity
</code></pre></div></div>

<h2 id="10-attributes">10. Attributes</h2>
<p>Attributes are the properties of a resource. They can be:</p>
<ol>
  <li>Required attributes: must be defined in the resource block.</li>
  <li>Optional attributes: can be defined but not required.</li>
</ol>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web_server1"</span> <span class="p">{</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span> <span class="c1"># Required attribute</span>
  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span> <span class="c1"># Optional attribute</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"Web Server 1"</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<p>In the above examples, <code class="language-plaintext highlighter-rouge">instance_type</code> is required attribute, and <code class="language-plaintext highlighter-rouge">tags</code> is optional attributes.</p>

<h2 id="11-cross-referencing-resource-attribute">11. Cross Referencing Resource Attribute</h2>
<p>Cross referencing is used to connect one resource to another.</p>

<p>Instead of writing value manually, we take the value from another resource.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_security_group"</span> <span class="s2">"web_sg"</span> <span class="p">{</span>
  <span class="nx">name</span> <span class="p">=</span> <span class="s2">"web-sg"</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web_server"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>
  <span class="nx">vpc_security_group_ids</span> <span class="p">=</span> <span class="p">[</span><span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">web_sg</span><span class="err">.</span><span class="nx">id</span><span class="p">]</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Here, the EC2 instance uses the ID from the security group. Terraform knows that EC2 depends on the security group.</p>

<h2 id="12-string-interpolcation">12. String Interpolcation</h2>
<p>String interpolation is used to insert a value inside a string.</p>

<p>Terraform uses this format:</p>
<pre><code class="language-code">${resource_type.resource_name.attribute}
</code></pre>
<p>Example</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">tags</span> <span class="err">=</span> <span class="p">{</span>
  <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"web-${aws_instance.web_server.id}"</span>
<span class="p">}</span>
</code></pre></div></div>

<p>This will create a tag with the EC2 instance ID inside the name. It’s like <code class="language-plaintext highlighter-rouge">web-i-0123456789abcdef0</code>.</p>

<h2 id="13-output-values">13. Output Values</h2>
<p>Output values are used to show information after terraform apply.</p>

<p>It is useful to see things like public IP, instance ID, etc.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">output</span> <span class="s2">"instance_public_ip"</span> <span class="p">{</span>
  <span class="nx">value</span> <span class="p">=</span> <span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web_server</span><span class="err">.</span><span class="nx">public_ip</span>
<span class="p">}</span>
</code></pre></div></div>

<p>After apply, Terraform will show the public IP in the terminal.</p>

<p>Output is useful when you need important information from your infrastructure.</p>

<h2 id="14-terraform-variables">14. Terraform Variables</h2>
<p>Terraform variables are used to make your code more flexible.</p>

<p>Instead of hardcoding values, you can use variables.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"instance_type"</span> <span class="p">{</span>
  <span class="nx">description</span> <span class="p">=</span> <span class="s2">"EC2 instance type"</span>
  <span class="nx">type</span>        <span class="p">=</span> <span class="nx">string</span>
  <span class="nx">default</span>     <span class="p">=</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>

<span class="c1"># Use the variable inside resource:</span>
<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web_server"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="nx">var</span><span class="err">.</span><span class="nx">instance_type</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Now we can change the instance type without editing the resource block.</p>

<p>Btw, if you have not defined a value for a variable. Terraform will ask you to input the value in CLI Terminal when run tf plan / apply.</p>

<h3 id="141-declaring-variable-values">14.1 Declaring Variable Values</h3>
<p>There are 4 ways to declare or assign variable values.</p>

<h4 id="1411-set-a-default-value-when-defining-the-variable-in-variablestf">14.1.1 Set a default value when defining the variable in <code class="language-plaintext highlighter-rouge">variables.tf</code></h4>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"instance_type"</span> <span class="p">{</span>
  <span class="nx">description</span> <span class="p">=</span> <span class="s2">"EC2 instance type"</span>
  <span class="nx">type</span>        <span class="p">=</span> <span class="nx">string</span>
  <span class="nx">default</span>     <span class="p">=</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>
</code></pre></div></div>

<h4 id="1412-using-tfvars-file">14.1.2 Using <code class="language-plaintext highlighter-rouge">.tfvars</code> File</h4>

<p>Create a file like <code class="language-plaintext highlighter-rouge">terraform.tfvars</code> and inside the file put <code class="language-plaintext highlighter-rouge">instance_type = "t3.small"</code>. Terraform will automatically read <code class="language-plaintext highlighter-rouge">terraform.tfvars</code>.</p>

<h4 id="1413-using-command-line--var">14.1.3 Using Command Line (<code class="language-plaintext highlighter-rouge">-var</code>)</h4>

<p>You can pass variable directly when running Terraform.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform apply <span class="nt">-var</span><span class="o">=</span><span class="s2">"instance_type=t3.small"</span>
</code></pre></div></div>

<h4 id="1413-using-environment-variables">14.1.3 Using Environment Variables</h4>

<p>You can set environment variable with the format <code class="language-plaintext highlighter-rouge">TF_VAR_&lt;variable_name&gt;</code>.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">export </span><span class="nv">TF_VAR_instance_type</span><span class="o">=</span><span class="s2">"t3.small"</span>
</code></pre></div></div>

<h2 id="15-variable-definiations-file-tfvars">15. Variable Definiations File (TFVars)</h2>
<p><code class="language-plaintext highlighter-rouge">.tfvars</code> file is used to assign values to variables.</p>

<p>Example:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform.tfvars
</code></pre></div></div>
<p>Inside the file:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">instance_type</span> <span class="err">=</span> <span class="s2">"t3.small"</span>
</code></pre></div></div>

<p>When you run terraform apply, it will automatically read the values from <code class="language-plaintext highlighter-rouge">terraform.tfvars</code>.</p>

<p>We can also use custom file:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform apply <span class="nt">-var-file</span><span class="o">=</span><span class="s2">"custom.tfvars"</span>

terraform apply <span class="nt">-var-file</span><span class="o">=</span><span class="s2">"dev.tfvars"</span>
</code></pre></div></div>

<p class="notice--info"><strong>Notes!</strong> If file name is terraform.tfvars -&gt; Terraform will automatically load values from it. If file name is different like prod.tfvars -&gt; We have to explicitly define the file during plan / apply operation.</p>

<h2 id="16-variable-definition-precedence">16. Variable Definition Precedence</h2>
<p>Variable definition precedence means the order Terraform uses when multiple variable values are provided.</p>

<p>If the same variable is defined in many places, Terraform will choose the value with the highest priority.</p>

<p>From lowest to highest precedence:</p>

<ol>
  <li>Default value inside <code class="language-plaintext highlighter-rouge">variable</code> block</li>
  <li>Environment variable (<code class="language-plaintext highlighter-rouge">TF_VAR_name</code>)</li>
  <li><code class="language-plaintext highlighter-rouge">terraform.tfvars</code> file</li>
  <li><code class="language-plaintext highlighter-rouge">-var-file</code> flag</li>
  <li><code class="language-plaintext highlighter-rouge">-var</code> flag (highest priority)</li>
</ol>

<p>Example</p>

<p>If you have:</p>
<ul>
  <li>Default → <code class="language-plaintext highlighter-rouge">t3.micro</code></li>
  <li>terraform.tfvars → <code class="language-plaintext highlighter-rouge">t3.small</code></li>
  <li>CLI <code class="language-plaintext highlighter-rouge">-var="instance_type=t3.large"</code></li>
</ul>

<p>Terraform will use <code class="language-plaintext highlighter-rouge">t3.large</code> because <code class="language-plaintext highlighter-rouge">-var</code> has the highest precedence.</p>

<h2 id="17-data-types">17. Data Types</h2>
<p>Terraform variables and arguments support different data types. Data types define what kind of value a variable can store.</p>

<h3 id="171-string">17.1 String</h3>
<p>A string is text.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"instance_type"</span> <span class="p">{</span>
  <span class="nx">type</span> <span class="p">=</span> <span class="nx">string</span>
<span class="p">}</span>

<span class="c1"># Value Example</span>
<span class="nx">instance_type</span> <span class="err">=</span> <span class="s2">"t3.micro"</span>
</code></pre></div></div>

<h3 id="172-number">17.2 Number</h3>
<p>A number is a numeric value.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"instance_count"</span> <span class="p">{</span>
  <span class="nx">type</span> <span class="p">=</span> <span class="nx">number</span>
<span class="p">}</span>

<span class="c1"># Value Example</span>
<span class="nx">instance_count</span> <span class="err">=</span> <span class="mi">3</span>
</code></pre></div></div>

<h3 id="173-boolean">17.3 Boolean</h3>
<p>A boolean is a true or false value.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"enable_monitoring"</span> <span class="p">{</span>
  <span class="nx">type</span> <span class="p">=</span> <span class="nx">bool</span>
<span class="p">}</span>

<span class="c1"># Value Example</span>
<span class="nx">enable_monitoring</span> <span class="err">=</span> <span class="kc">true</span>
</code></pre></div></div>

<h3 id="174-list">17.4 List</h3>
<p>A list is an ordered collection of values.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"availability_zones"</span> <span class="p">{</span>
  <span class="nx">type</span> <span class="p">=</span> <span class="nx">list</span><span class="err">(</span><span class="nx">string</span><span class="err">)</span>
<span class="p">}</span>

<span class="c1"># Value Example</span>
<span class="nx">availability_zones</span> <span class="err">=</span> <span class="p">[</span><span class="s2">"us-east-1a"</span><span class="p">,</span> <span class="s2">"us-east-1b"</span><span class="p">]</span>
</code></pre></div></div>

<h3 id="175-map">17.5 Map</h3>
<p>A map is a collection of key-value pairs.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"tags"</span> <span class="p">{</span>
  <span class="nx">type</span> <span class="p">=</span> <span class="nx">map</span><span class="err">(</span><span class="nx">string</span><span class="err">)</span>
<span class="p">}</span>

<span class="c1"># Value Example</span>
<span class="nx">tags</span> <span class="err">=</span> <span class="p">{</span>
  <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"Web Server"</span>
  <span class="nx">Env</span>  <span class="p">=</span> <span class="s2">"Production"</span>
<span class="p">}</span>
</code></pre></div></div>

<h3 id="176-null">17.6 Null</h3>
<p>Null is a special value that represents the absence of a value.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"instance_type"</span> <span class="p">{</span>
  <span class="nx">type</span>    <span class="p">=</span> <span class="nx">string</span>
  <span class="nx">default</span> <span class="p">=</span> <span class="kc">null</span>
<span class="p">}</span>

<span class="c1"># Value Example</span>
<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="nx">var</span><span class="err">.</span><span class="nx">instance_type</span>
<span class="p">}</span>
</code></pre></div></div>

<p>In this example, if <code class="language-plaintext highlighter-rouge">instance_type</code> is not defined, it will be null. Terraform will use the default instance type for the resource.</p>

<h2 id="18-the-count-meta-argument">18. The COUNT Meta-Argument</h2>
<p>The <code class="language-plaintext highlighter-rouge">count</code> meta-argument is used to create multiple resources from a single resource block.</p>

<p>Instead of writing the same resource many times, we can use <code class="language-plaintext highlighter-rouge">count</code>.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web_server"</span> <span class="p">{</span>
  <span class="nx">count</span>         <span class="p">=</span> <span class="mi">3</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>
</code></pre></div></div>

<p>This will create 3 EC2 instances.</p>

<h3 id="181-accessing-each-resource">18.1 Accessing Each Resource</h3>

<p>Terraform gives each resource an index number starting from 0.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web_server</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web_server</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web_server</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
</code></pre></div></div>

<h3 id="181-using-countindex">18.1 Using <code class="language-plaintext highlighter-rouge">count.index</code></h3>
<p>You can use <code class="language-plaintext highlighter-rouge">count.index</code> to create different values for each resource.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web_server"</span> <span class="p">{</span>
  <span class="nx">count</span>         <span class="p">=</span> <span class="mi">3</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"web-server-${count.index}"</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Results:</p>
<ul>
  <li>web-server-0</li>
  <li>web-server-1</li>
  <li>web-server-2</li>
</ul>

<p>Btw if you’re using <code class="language-plaintext highlighter-rouge">count</code> when creating IAM User it will not work since we cannot have multiple IAM User with the exact same name. So, the solution should use <code class="language-plaintext highlighter-rouge">count.index</code>.</p>

<h2 id="19-conditional-expressions">19. Conditional Expressions</h2>
<p>Conditional expressions are used to set a value based on a condition.</p>

<p>Terraform uses a simple <strong>if–else style</strong> expression.</p>

<p>Format:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">condition</span> <span class="err">?</span> <span class="nx">true_value</span> <span class="err">:</span> <span class="nx">false_value</span>
</code></pre></div></div>

<ul>
  <li>If the condition is <code class="language-plaintext highlighter-rouge">true</code>, Terraform uses <code class="language-plaintext highlighter-rouge">true_value</code>.</li>
  <li>If the condition is <code class="language-plaintext highlighter-rouge">false</code>, Terraform uses <code class="language-plaintext highlighter-rouge">false_value</code>.</li>
</ul>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"environment"</span> <span class="p">{</span>
  <span class="nx">type</span>    <span class="p">=</span> <span class="nx">string</span>
  <span class="nx">default</span> <span class="p">=</span> <span class="s2">"dev"</span>
<span class="p">}</span>

<span class="c1"># Use conditional expression</span>
<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web_server"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="nx">var</span><span class="err">.</span><span class="nx">environment</span> <span class="p">==</span> <span class="s2">"prod"</span> <span class="err">?</span> <span class="s2">"t3.large"</span> <span class="err">:</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Explanation:</p>
<ul>
  <li>If environment = <code class="language-plaintext highlighter-rouge">prod</code> → instance type <code class="language-plaintext highlighter-rouge">t3.medium</code></li>
  <li>Otherwise → instance type <code class="language-plaintext highlighter-rouge">t3.micro</code></li>
</ul>

<h3 id="191-using-multiple-variables">19.1 Using Multiple Variables</h3>
<p>We can also use multiple variables in a conditional expression.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"environment"</span> <span class="p">{</span>
  <span class="nx">type</span> <span class="p">=</span> <span class="nx">string</span>
<span class="p">}</span>

<span class="nx">variable</span> <span class="s2">"high_availability"</span> <span class="p">{</span>
  <span class="nx">type</span> <span class="p">=</span> <span class="nx">bool</span>
<span class="p">}</span>

<span class="c1"># Use both variables</span>
<span class="nx">count</span> <span class="err">=</span> <span class="nx">var</span><span class="err">.</span><span class="nx">environment</span> <span class="err">==</span> <span class="s2">"prod"</span> <span class="err">&amp;&amp;</span> <span class="nx">var</span><span class="err">.</span><span class="nx">high_availability</span> <span class="err">?</span> <span class="mi">3</span> <span class="err">:</span> <span class="mi">1</span>
</code></pre></div></div>

<p>Explanation:</p>
<ul>
  <li>If environment is <code class="language-plaintext highlighter-rouge">prod</code> and high availability is <code class="language-plaintext highlighter-rouge">true</code> → create 3 instances</li>
  <li>Otherwise → create 1 instance</li>
</ul>

<h2 id="20-terraform-functions">20. Terraform Functions</h2>
<p>Terraform functions are built-in functions that help us transform or calculate values in our configuration.</p>

<p>We can use functions to manipulate strings, numbers, lists, files, and more.</p>

<p>Example</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Using the upper function to convert a string to uppercase</span>
<span class="nx">variable</span> <span class="s2">"environment"</span> <span class="p">{</span>
  <span class="nx">type</span> <span class="p">=</span> <span class="nx">string</span>
<span class="p">}</span>
<span class="nx">output</span> <span class="s2">"env_upper"</span> <span class="p">{</span>
  <span class="nx">value</span> <span class="p">=</span> <span class="nx">upper</span><span class="err">(</span><span class="nx">var</span><span class="err">.</span><span class="nx">environment</span><span class="err">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This will take the value of <code class="language-plaintext highlighter-rouge">environment</code> variable and convert it to uppercase.</p>

<p><strong>General Format</strong></p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>function_name<span class="o">(</span>argument1, argument2<span class="o">)</span>
</code></pre></div></div>

<p><strong><code class="language-plaintext highlighter-rouge">max()</code> Function</strong></p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">locals</span> <span class="p">{</span>
  <span class="nx">highest_number</span> <span class="p">=</span> <span class="nx">max</span><span class="err">(</span><span class="mi">5</span><span class="err">,</span> <span class="mi">10</span><span class="err">,</span> <span class="mi">20</span><span class="err">)</span>
<span class="p">}</span>

<span class="c1"># Terraform will  output 30 as the largest number</span>
<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"example"</span> <span class="p">{</span>
  <span class="nx">count</span> <span class="p">=</span> <span class="nx">max</span><span class="err">(</span><span class="mi">1</span><span class="err">,</span> <span class="mi">2</span><span class="err">,</span> <span class="mi">3</span><span class="err">)</span>
<span class="p">}</span>

</code></pre></div></div>

<p>Terraform will create 3 instances because 3 is the largest value.</p>

<p><strong><code class="language-plaintext highlighter-rouge">file()</code> Function</strong></p>

<p>The file() function reads the content of a file from our local system.</p>

<p>It can be used to put the <code class="language-plaintext highlighter-rouge">user data</code> or <code class="language-plaintext highlighter-rouge">policy.json</code>.</p>

<p>Example we have file <code class="language-plaintext highlighter-rouge">user_data.sh</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
<span class="nb">echo</span> <span class="s2">"Hello Terraform"</span>
</code></pre></div></div>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web_server"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>

  <span class="nx">user_data</span> <span class="p">=</span> <span class="nx">file</span><span class="err">(</span><span class="s2">"user_data.sh"</span><span class="err">)</span>
<span class="p">}</span>
</code></pre></div></div>

<p>This is useful when we want to pass the user data without including it to the scripts.</p>

<h2 id="21-local-values">21. Local Values</h2>
<p>Local values (locals) are used to store temporary values inside a Terraform configuration.</p>

<p>It is used for reusing values. So, instead of repeating the same value many times, you can define it once and use it everywhere.</p>

<p>Here are few example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">locals</span> <span class="p">{</span>
  <span class="nx">environment</span> <span class="p">=</span> <span class="s2">"production"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>
  <span class="nx">instance_name</span> <span class="p">=</span> <span class="s2">"web-server"</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web_server"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="nx">local</span><span class="err">.</span><span class="nx">instance_type</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Env</span> <span class="p">=</span> <span class="nx">local</span><span class="err">.</span><span class="nx">environment</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="nx">local</span><span class="err">.</span><span class="nx">instance_name</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Another Example:</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">locals</span> <span class="p">{</span>
  <span class="nx">common_tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Environment</span> <span class="p">=</span> <span class="s2">"dev"</span>
    <span class="nx">Owner</span>       <span class="p">=</span> <span class="s2">"team-devops"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="nx">local</span><span class="err">.</span><span class="nx">common_tags</span>
<span class="p">}</span>
</code></pre></div></div>

<h2 id="data-sources">Data Sources</h2>
<p>Data sources are used to get information from existing resources.</p>

<p>Instead of creating a new resource, Terraform reads data from resources that already exist in the cloud.</p>

<p><strong>Example: Read Existing AWS Resource</strong></p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">data</span> <span class="s2">"aws_ami"</span> <span class="s2">"amazon_linux"</span> <span class="p">{</span>
  <span class="nx">most_recent</span> <span class="p">=</span> <span class="kc">true</span>

  <span class="nx">owners</span> <span class="p">=</span> <span class="p">[</span><span class="s2">"amazon"</span><span class="p">]</span>

  <span class="nx">filter</span> <span class="p">{</span>
    <span class="nx">name</span>   <span class="p">=</span> <span class="s2">"name"</span>
    <span class="nx">values</span> <span class="p">=</span> <span class="p">[</span><span class="s2">"amzn2-ami-hvm-*"</span><span class="p">]</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This will get the most recent Amazon Linux 2 AMI ID from AWS.</p>

<p>To use it in resources:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="nx">data</span><span class="err">.</span><span class="nx">aws_ami</span><span class="err">.</span><span class="nx">amazon_linux</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>
</code></pre></div></div>

<p><strong>Example: Read Local File</strong></p>

<p>Let’s say we have file <code class="language-plaintext highlighter-rouge">user_data.sh</code> in our current file system path where the tf scripts is located. We can fetch using data sources:</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">data</span> <span class="s2">"local_file"</span> <span class="s2">"user_data"</span> <span class="p">{</span>
  <span class="nx">filename</span> <span class="p">=</span> <span class="s2">"${path.module}/user_data.sh"</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This will read the content of <code class="language-plaintext highlighter-rouge">user_data.sh</code> file and we can use it in our resource.</p>

<p>There are so much list available data source in the provider, like AWS. Visit Terraform documentation for that.</p>

<p>By using the data sources, it solve a problem to use static information such as put hard coded value of AMI ID to the EC2 resource.</p>

<h2 id="terraform-fmt-and-validate">Terraform <code class="language-plaintext highlighter-rouge">fmt</code> and <code class="language-plaintext highlighter-rouge">validate</code></h2>
<p><code class="language-plaintext highlighter-rouge">terraform fmt</code> is used to format our Terraform code.</p>

<p>It will automatically fix spacing, indentation, and layout.</p>

<p>For example, we have code that the format is inconsistent and doesn’t follow terraform standard.</p>

<p>We just have to pass command <code class="language-plaintext highlighter-rouge">terraform fmt</code> and the code will be clean and easier to read.</p>

<p>Btw in VS Code, there is an extension terrafom that will automatically format the code if we save the file.</p>

<p>Now, for <code class="language-plaintext highlighter-rouge">terraform validate</code> is used to check if our configuration is valid.</p>

<p>It checks syntax and basic errors, but it does not create resources.</p>

<p>We just have to pass command <code class="language-plaintext highlighter-rouge">terraform validate</code> before running <code class="language-plaintext highlighter-rouge">terraform plan</code>.</p>

<h2 id="dynamic-block">Dynamic Block</h2>
<p>Dynamic block is used to create repeated nested blocks inside a resource.</p>

<p>It is useful when we want to avoid writing the same block many times.</p>

<p>The reason we use dynamic blocks is that sometimes a resource needs multiple nested blocks, like multiple security group rules.</p>

<p>Instead of writing them one by one, we can use a dynamic block.</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"ports"</span> <span class="p">{</span>
  <span class="nx">default</span> <span class="p">=</span> <span class="p">[</span><span class="mi">80</span><span class="p">,</span> <span class="mi">443</span><span class="p">]</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_security_group"</span> <span class="s2">"web_sg"</span> <span class="p">{</span>
  <span class="nx">name</span> <span class="p">=</span> <span class="s2">"web-sg"</span>

  <span class="nx">dynamic</span> <span class="s2">"ingress"</span> <span class="p">{</span>
    <span class="nx">for_each</span> <span class="p">=</span> <span class="nx">var</span><span class="err">.</span><span class="nx">ports</span>
    <span class="nx">content</span> <span class="p">{</span>
      <span class="nx">from_port</span>   <span class="p">=</span> <span class="nx">ingress</span><span class="err">.</span><span class="nx">value</span>
      <span class="nx">to_port</span>     <span class="p">=</span> <span class="nx">ingress</span><span class="err">.</span><span class="nx">value</span>
      <span class="nx">protocol</span>    <span class="p">=</span> <span class="s2">"tcp"</span>
      <span class="nx">cidr_blocks</span> <span class="p">=</span> <span class="p">[</span><span class="s2">"0.0.0.0/0"</span><span class="p">]</span>
    <span class="p">}</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Explanation</p>
<ul>
  <li><code class="language-plaintext highlighter-rouge">dynamic "ingress"</code> → create multiple ingress blocks</li>
  <li><code class="language-plaintext highlighter-rouge">for_each</code> → loop over the list</li>
  <li><code class="language-plaintext highlighter-rouge">ingress.value</code> → current value (port)</li>
</ul>

<p>This will create:</p>
<ul>
  <li>One rule for <code class="language-plaintext highlighter-rouge">port 80</code></li>
  <li>One rule for <code class="language-plaintext highlighter-rouge">port 443</code></li>
</ul>

<h2 id="terraform-replace">Terraform Replace</h2>
<p><code class="language-plaintext highlighter-rouge">terraform replace</code> is used to force recreate a resource.</p>

<p>It tells Terraform to destroy and create the resource again, even if there is no change in the code.</p>

<p>For example, if the EC2 instance has an issue and we want a fresh one:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform apply <span class="nt">-replace</span><span class="o">=</span><span class="s2">"aws_instance.web_server"</span>
</code></pre></div></div>

<p>Terraform will destroy the existing instance and create a new one.</p>

<p>When to Use:</p>
<ul>
  <li>Resource is broken</li>
  <li>We want to recreate without changing code</li>
  <li>Replace old resource with new one</li>
</ul>

<h2 id="saving-terraform-plan-to-file">Saving Terraform Plan to File</h2>
<p>Terraform allows to save plan file.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform plan <span class="nt">-out</span> infra.plan
</code></pre></div></div>

<p>To apply from plan file.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform apply infra.plan
</code></pre></div></div>

<h2 id="resource-targetting">Resource Targetting</h2>
<p>Resource targeting is used to apply changes to specific resources only.</p>

<p>Instead of running Terraform on all resources, we can focus on one resource.</p>

<p>Format:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform apply <span class="nt">-target</span><span class="o">=</span>&lt;resource_type&gt;.&lt;resource_name&gt;
</code></pre></div></div>

<p>Example:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform apply <span class="nt">-target</span><span class="o">=</span>aws_instance.web_server
</code></pre></div></div>

<p>This will only create or update to <code class="language-plaintext highlighter-rouge">aws_instance.web_server</code> and ignore other resources.</p>

<p>We can also destroy specific resource using <code class="language-plaintext highlighter-rouge">-target</code>.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform destroy <span class="nt">-target</span><span class="o">=</span>aws_instance.web_server
</code></pre></div></div>

<p>This will only destroy <code class="language-plaintext highlighter-rouge">aws_instance.web_server</code> and ignore other resources.</p>

<h2 id="comments">Comments</h2>
<p>Comments are used to write notes in our code. Terraform will ignore comments when running.</p>

<h3 id="single-line-comment">Single Line Comment</h3>
<p>Use <code class="language-plaintext highlighter-rouge">#</code> or <code class="language-plaintext highlighter-rouge">//</code>.</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># This is a single line comment</span>
<span class="c1">// This is also a single line comment</span>
</code></pre></div></div>

<h3 id="multi-line-comment">Multi Line Comment</h3>
<p>Use <code class="language-plaintext highlighter-rouge">/* */</code>.</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/*
This is a multi line comment
It can span multiple lines
*/</span>
</code></pre></div></div>

<h2 id="resource-behavior-and-meta-argument">Resource Behavior and Meta-Argument</h2>
<p>A resource block means we want a resource to exist with a specific configuration.</p>

<p><strong>How Terraform Applies a Configuration</strong></p>
<ul>
  <li>Create resrouce that exist in the configuration (scripts) but are not associate with a real infrastructure object in the state</li>
  <li>Destroy resource that exist in the state (console) but no longer exist in the configuration (scripts)</li>
  <li>Update in-place resource if resource can be updated without recreation</li>
  <li>Destroy and re-create resources whose arguments have changed but which cannot be updated in-place due to remote API limitations and requires new resource</li>
</ul>

<p>There are different meta arguments</p>
<h3 id="lifecycle">lifecycle</h3>
<p><code class="language-plaintext highlighter-rouge">lifecycle</code> block is used to control how Terraform handles a resource.</p>

<h4 id="create_before_destroy">create_before_destroy</h4>
<p>Create new resource first, then delete the old one.</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">lifecycle</span> <span class="p">{</span>
  <span class="nx">create_before_destroy</span> <span class="p">=</span> <span class="kc">true</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Use case:</p>
<ul>
  <li>Avoid downtime when replacing a resource.</li>
</ul>

<h4 id="prevent_destroy">prevent_destroy</h4>
<p>Prevent resource from being deleted.</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">lifecycle</span> <span class="p">{</span>
  <span class="nx">prevent_destroy</span> <span class="p">=</span> <span class="kc">true</span>
<span class="p">}</span>
</code></pre></div></div>

<p>If we try to destroy, Terraform will show error.</p>

<p>Use case:</p>
<ul>
  <li>Protect critical resources from accidental deletion (like database).</li>
</ul>

<h4 id="ignore_changes">ignore_changes</h4>
<p>Ignore specific changes in a resource. Sometimes we use to ignore changes in the tags.</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">lifecycle</span> <span class="p">{</span>
  <span class="nx">ignore_changes</span> <span class="p">=</span> <span class="p">[</span><span class="nx">tags</span><span class="p">]</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Use case:</p>
<ul>
  <li>Ignore changes that are made outside of Terraform (like manual changes in AWS Console).</li>
</ul>

<h4 id="replace_triggered_by">replace_triggered_by</h4>
<p>Force resource to be replaced when something changes.</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">lifecycle</span> <span class="p">{</span>
  <span class="nx">replace_triggered_by</span> <span class="p">=</span> <span class="p">[</span><span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">web_sg</span><span class="p">]</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Use case:</p>
<ul>
  <li>Force replacement when a related resource changes (like security group).</li>
</ul>

<h3 id="depends_on">depends_on</h3>
<p><code class="language-plaintext highlighter-rouge">depends_on</code> is used to tell Terraform that one resource depends on another resource.</p>

<p>It ensures Terraform creates resources in the correct order.</p>

<p>Terraform usually detects dependencies automatically.</p>

<p>But sometimes, the dependency is not clear.</p>

<p>In that case, we need to define it manually using <code class="language-plaintext highlighter-rouge">depends_on</code>.</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_security_group"</span> <span class="s2">"web_sg"</span> <span class="p">{</span>
  <span class="nx">name</span> <span class="p">=</span> <span class="s2">"web-sg"</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>

  <span class="nx">depends_on</span> <span class="p">=</span> <span class="p">[</span><span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">web_sg</span><span class="p">]</span>
<span class="p">}</span>
</code></pre></div></div>

<p>In this example, terraform will create security group first and then create the ec2.</p>

<p><strong>When to Use:</strong></p>
<ul>
  <li>When dependency is not obvious</li>
  <li>When resource uses indirect dependency</li>
  <li>When Terraform does not detect dependency automatically</li>
</ul>

<h3 id="count">count</h3>
<p><code class="language-plaintext highlighter-rouge">count</code> is used to create multiple resources from a single resource block.</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">count</span>         <span class="p">=</span> <span class="mi">3</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>

<span class="c1"># Access Each Resource</span>
<span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
</code></pre></div></div>

<p>Terraform will create 3 EC2 instances.</p>

<p><strong>Conditional count</strong></p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">count</span> <span class="err">=</span> <span class="nx">var</span><span class="err">.</span><span class="nx">environment</span> <span class="err">==</span> <span class="s2">"prod"</span> <span class="err">?</span> <span class="mi">3</span> <span class="err">:</span> <span class="mi">1</span>
</code></pre></div></div>

<ul>
  <li>If <code class="language-plaintext highlighter-rouge">prod</code> → 3 instances</li>
  <li>If not → 1 instance</li>
</ul>

<h3 id="for_each">for_each</h3>
<p><code class="language-plaintext highlighter-rouge">for_each</code> is used to create multiple resources using a map or set.</p>

<p>It is similar to <code class="language-plaintext highlighter-rouge">count</code>, but gives more control and better naming.</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">for_each</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">dev</span>  <span class="p">=</span> <span class="s2">"t3.micro"</span>
    <span class="nx">prod</span> <span class="p">=</span> <span class="s2">"t3.medium"</span>
  <span class="p">}</span>

  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="nx">each</span><span class="err">.</span><span class="nx">value</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="nx">each</span><span class="err">.</span><span class="nx">key</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Accessing Resource</span>
<span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web</span><span class="p">[</span><span class="s2">"dev"</span><span class="p">]</span>
<span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web</span><span class="p">[</span><span class="s2">"prod"</span><span class="p">]</span>

</code></pre></div></div>
<p>This will create:</p>
<ul>
  <li><code class="language-plaintext highlighter-rouge">dev</code> instance with type <code class="language-plaintext highlighter-rouge">t3.micro</code></li>
  <li><code class="language-plaintext highlighter-rouge">prod</code> instance with type <code class="language-plaintext highlighter-rouge">t3.medium</code></li>
</ul>

<p><strong>Difference with count</strong></p>
<ul>
  <li><code class="language-plaintext highlighter-rouge">count</code> → uses index (0, 1, 2)</li>
  <li><code class="language-plaintext highlighter-rouge">for_each</code> → uses key (dev, prod)</li>
</ul>

<h3 id="provider">provider</h3>
<p><code class="language-plaintext highlighter-rouge">provider</code> meta-argument is used to specify which provider configuration a resource should use.</p>

<p>This is useful when we have multiple providers (like different regions or accounts).</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">provider</span> <span class="s2">"aws"</span> <span class="p">{</span>
  <span class="nx">region</span> <span class="p">=</span> <span class="s2">"us-east-1"</span>
<span class="p">}</span>

<span class="nx">provider</span> <span class="s2">"aws"</span> <span class="p">{</span>
  <span class="nx">alias</span>  <span class="p">=</span> <span class="s2">"west"</span>
  <span class="nx">region</span> <span class="p">=</span> <span class="s2">"us-west-2"</span>
<span class="p">}</span>


<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">provider</span>      <span class="p">=</span> <span class="nx">aws</span><span class="err">.</span><span class="nx">west</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This will create:</p>
<ul>
  <li>Default provider → <code class="language-plaintext highlighter-rouge">us-east-1</code></li>
  <li>This resource → uses <code class="language-plaintext highlighter-rouge">us-west-2</code></li>
</ul>

<p><strong>When to Use</strong></p>
<ul>
  <li>Multiple regions</li>
  <li>Multiple AWS accounts</li>
  <li>Different configurations</li>
</ul>

<h2 id="terraform-provisioners">Terraform Provisioners</h2>
<p>Provisioners are used to run scripts or commands after a resource is created.</p>

<p>They are usually used for setup tasks, like installing software.</p>

<h3 id="local-exec">local-exec</h3>

<p><code class="language-plaintext highlighter-rouge">local-exec</code> runs a command on our local machine (where we run Terraform).</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>

  <span class="nx">provisioner</span> <span class="s2">"local-exec"</span> <span class="p">{</span>
    <span class="nx">command</span> <span class="p">=</span> <span class="s2">"echo Instance created"</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<p>This will print “Instance created” in our terminal after the EC2 instance is created.</p>

<h3 id="remote-exec">remote-exec</h3>

<p><code class="language-plaintext highlighter-rouge">remote-exec</code> runs commands inside the resource (like inside EC2).</p>

<p>Example:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>

  <span class="nx">connection</span> <span class="p">{</span>
    <span class="nx">type</span>        <span class="p">=</span> <span class="s2">"ssh"</span>
    <span class="nx">user</span>        <span class="p">=</span> <span class="s2">"ec2-user"</span>
    <span class="nx">private_key</span> <span class="p">=</span> <span class="nx">file</span><span class="err">(</span><span class="s2">"key.pem"</span><span class="err">)</span>
    <span class="nx">host</span>        <span class="p">=</span> <span class="nx">self</span><span class="err">.</span><span class="nx">public_ip</span>
  <span class="p">}</span>

  <span class="nx">provisioner</span> <span class="s2">"remote-exec"</span> <span class="p">{</span>
    <span class="nx">inline</span> <span class="p">=</span> <span class="p">[</span>
      <span class="s2">"sudo yum update -y"</span><span class="p">,</span>
      <span class="s2">"sudo yum install -y nginx"</span>
    <span class="p">]</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<p>This will connect to the EC2 instance via SSH and run the commands to update and install Nginx.</p>

<h2 id="terraform-modules">Terraform Modules</h2>
<p>Modules are used to organize and re-use code in Terraform.</p>

<p>Instead of writing everything in one file, we can split our code into smaller parts (modules).</p>

<p><strong>Why Use Modules?</strong></p>
<ul>
  <li>Make code cleaner</li>
  <li>Reuse the same configuration</li>
  <li>Easier to manage large projects</li>
</ul>

<h3 id="example-modules">Example Modules</h3>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>modules/ec2/main.tf
</code></pre></div></div>

<p>Inside the module:</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>
</code></pre></div></div>

<p>To use module in <code class="language-plaintext highlighter-rouge">main.tf</code>:</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">module</span> <span class="s2">"ec2_instance"</span> <span class="p">{</span>
  <span class="nx">source</span> <span class="p">=</span> <span class="s2">"./modules/ec2"</span>
<span class="p">}</span>
</code></pre></div></div>

<h3 id="modules-with-variables">Modules with Variables</h3>

<p><strong>Module</strong>:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"instance_type"</span> <span class="p">{}</span>

<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="nx">var</span><span class="err">.</span><span class="nx">instance_type</span>
<span class="p">}</span>
</code></pre></div></div>

<p><strong>Main.tf</strong>:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">module</span> <span class="s2">"ec2_instance"</span> <span class="p">{</span>
  <span class="nx">source</span>        <span class="p">=</span> <span class="s2">"./modules/ec2"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>
</code></pre></div></div>

<p>This allows us to pass different values to the module when we use it.</p>

<p><strong>Point to Notes</strong></p>
<ul>
  <li>Module = reusable Terraform code</li>
  <li><code class="language-plaintext highlighter-rouge">source</code> = where module is located</li>
  <li>We can pass variables into module</li>
</ul>

<h3 id="avoid-hardcoded-values-in-variables">Avoid Hardcoded Values in Variables</h3>
<p>Avoid writing fixed values directly in the Terraform code.</p>

<p>Hardcoded values make our code less flexible and harder to reuse.</p>

<h4 id="bad-examples-hardcoded">Bad Examples (Hardcoded)</h4>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>
</code></pre></div></div>

<h4 id="good-example-using-variables">Good Example (Using Variables)</h4>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"instance_type"</span> <span class="p">{</span>
  <span class="nx">default</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>
<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="nx">var</span><span class="err">.</span><span class="nx">instance_type</span>
<span class="p">}</span>
</code></pre></div></div>

<p>This way, we can change the instance type without editing the resource block. We can also reuse the same code for different instance types by passing different values to the variable.</p>

<h2 id="module-outputs">Module Outputs</h2>
<p>Module outputs are used to return values from a module.</p>

<p>So we can use that value in our main configuration.</p>

<p><strong>Example</strong></p>

<p>Inside module:</p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">ami</span>           <span class="p">=</span> <span class="s2">"ami-123456"</span>
  <span class="nx">instance_type</span> <span class="p">=</span> <span class="s2">"t3.micro"</span>
<span class="p">}</span>

<span class="nx">output</span> <span class="s2">"instance_id"</span> <span class="p">{</span>
  <span class="nx">value</span> <span class="p">=</span> <span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web</span><span class="err">.</span><span class="nx">id</span>
<span class="p">}</span>
</code></pre></div></div>

<p><strong>Use Output in Main File</strong></p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">module</span> <span class="s2">"ec2_instance"</span> <span class="p">{</span>
  <span class="nx">source</span> <span class="p">=</span> <span class="s2">"./modules/ec2"</span>
<span class="p">}</span>

<span class="c1"># Accessing the output</span>
<span class="nx">output</span> <span class="s2">"my_instance_id"</span> <span class="p">{</span>
  <span class="nx">value</span> <span class="p">=</span> <span class="nx">module</span><span class="err">.</span><span class="nx">ec2_instance</span><span class="err">.</span><span class="nx">instance_id</span>
<span class="p">}</span>
</code></pre></div></div>

<p><strong>The flow is like:</strong></p>
<ul>
  <li>Module creates EC2</li>
  <li>Module returns instance_id</li>
  <li>Main file can use that value</li>
</ul>

<h2 id="terraform-backend">Terraform Backend</h2>
<p>Terraform backend is used to <strong>store the state file</strong>.</p>

<p>By default, Terraform stores state locally (terraform.tfstate).</p>

<p>With backend, we can store it remotely.</p>

<p><strong>Why Use Backend?</strong></p>
<ul>
  <li>Share state with team</li>
  <li>Keep state safe</li>
  <li>Avoid conflicts</li>
</ul>

<h3 id="example-s3-backend">Example: S3 Backend</h3>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">terraform</span> <span class="p">{</span>
  <span class="nx">backend</span> <span class="s2">"s3"</span> <span class="p">{</span>
    <span class="nx">bucket</span> <span class="p">=</span> <span class="s2">"my-terraform-state"</span>
    <span class="nx">key</span>    <span class="p">=</span> <span class="s2">"dev/terraform.tfstate"</span>
    <span class="nx">region</span> <span class="p">=</span> <span class="s2">"us-east-1"</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<p>This will store the state file in an S3 bucket.</p>

<p class="notice--danger"><strong>Notes!</strong> It will be better to have S3 versioning enabled to allow recovery in case of accidental deletions.</p>

<h2 id="terraform-state-management">Terraform State Management</h2>
<p>Terraform state commands are used to manage and inspect the state file.</p>

<h3 id="state-list">state list</h3>
<p>Show all resources in the state.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform state list
</code></pre></div></div>

<h3 id="state-show">state show</h3>
<p>Show detailed information about a resource.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform state show aws_instance.web
</code></pre></div></div>

<h3 id="state-mv">state mv</h3>
<p>Move or rename a resource in the state.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform state <span class="nb">mv </span>aws_instance.old_name aws_instance.new_name
</code></pre></div></div>

<h3 id="state-rm">state rm</h3>
<p>Remove a resource from the state (does not delete real resource).</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform state <span class="nb">rm </span>aws_instance.web
</code></pre></div></div>

<h3 id="state-pull">state pull</h3>
<p>Download the current state file.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform state pull <span class="o">&gt;</span> terraform.tfstate
</code></pre></div></div>

<h3 id="state-push">state push</h3>
<p>Upload a state file manually.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform state push terraform.tfstate
</code></pre></div></div>

<h3 id="state-replace-provider">state replace-provider</h3>
<p>Replace provider in the state.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform state replace-provider hashicorp/aws registry.terraform.io/hashicorp/aws
</code></pre></div></div>

<h2 id="terraform-import">Terraform Import</h2>
<p><code class="language-plaintext highlighter-rouge">terraform import</code> is used to bring an existing resource into Terraform state.</p>

<p>This is useful when a resource was created manually (for example in AWS Console) and we want Terraform to manage it.</p>

<p><strong>What Import Does?</strong></p>
<ul>
  <li>Adds the existing resource into terraform.tfstate</li>
  <li>Lets Terraform track the resource</li>
</ul>

<p><strong>Example: Importing an EC2 Instance</strong></p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>terraform import aws_instance.web i-0123456789abcdef0
</code></pre></div></div>

<p>This will import the EC2 instance with ID <code class="language-plaintext highlighter-rouge">i-0123456789abcdef0</code> into Terraform state as <code class="language-plaintext highlighter-rouge">aws_instance.web</code>.</p>

<p><strong>Important Notes</strong></p>
<ul>
  <li>Import does not create a resource, it only adds existing resource to the state.</li>
  <li>After import, we need to write the corresponding resource block in your <code class="language-plaintext highlighter-rouge">.tf</code> files to match the imported resource.</li>
  <li>If the resource block does not match the imported resource, Terraform will show differences in <code class="language-plaintext highlighter-rouge">terraform plan</code> and may try to modify or recreate the resource.</li>
</ul>

<p>There is a way to generate tf scripts, using <code class="language-plaintext highlighter-rouge">-generate-config-out</code>. See the documentation for more details.</p>

<h2 id="sensitive-parameter">Sensitive Parameter</h2>
<p>Sensitive parameters are used to mark a variable or output as sensitive.</p>

<p>When a value is marked as sensitive, Terraform will not show it in the terminal output or logs.</p>

<p>This is useful for values like:</p>
<ul>
  <li>Password</li>
  <li>Access key</li>
  <li>Secret key</li>
  <li>Token</li>
</ul>

<p><strong>Example variable</strong></p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="s2">"db_password"</span> <span class="p">{</span>
  <span class="nx">type</span>      <span class="p">=</span> <span class="nx">string</span>
  <span class="nx">sensitive</span> <span class="p">=</span> <span class="kc">true</span>
<span class="p">}</span>
</code></pre></div></div>

<p><strong>Example Output</strong></p>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">output</span> <span class="s2">"db_password"</span> <span class="p">{</span>
  <span class="nx">value</span>     <span class="p">=</span> <span class="nx">var</span><span class="err">.</span><span class="nx">db_password</span>
  <span class="nx">sensitive</span> <span class="p">=</span> <span class="kc">true</span>
<span class="p">}</span>
</code></pre></div></div>

<p>When we run <code class="language-plaintext highlighter-rouge">terraform apply</code>, the value of <code class="language-plaintext highlighter-rouge">db_password</code> will not be shown in the terminal.</p>

<p><strong>Important Note</strong></p>
<ul>
  <li>Sensitive only hides the value in CLI output.</li>
  <li>The real value may still exist inside the state file.</li>
</ul>

<p>So we must keep our state file secure.</p>]]></content><author><name>Nugroho</name></author><category term="cloud" /><category term="aws" /><category term="terraform" /><category term="iac" /><summary type="html"><![CDATA[it’s just terraform notes.]]></summary></entry><entry><title type="html">Multi Attach EFS</title><link href="https://galnug.com/multi-attach-efs/" rel="alternate" type="text/html" title="Multi Attach EFS" /><published>2025-03-19T21:00:00+07:00</published><updated>2025-03-19T21:00:00+07:00</updated><id>https://galnug.com/multi-attach-efs</id><content type="html" xml:base="https://galnug.com/multi-attach-efs/"><![CDATA[<p>Amazon Elastic File System (EFS) is a fully managed, scalable file storage service designed to be shared across multiple compute resources including Amazon EC2, Amazon ECS, Amazon EKS, AWS Lambda, and AWS Fargate. Unlike Amazon EBS, which is typically attached to a single instance (with limited multi-attach support), EFS is built from the ground up to support multiple EC2 instances mounting the same filesystem simultaneously.</p>

<p>In this article, i’ll demonstrate how we can mount EFS to multiple instances.</p>

<h2 id="architecture-overview">Architecture Overview</h2>
<p>The following architecture is used to mount EFS to multiple EC2.</p>

<figure>
  
<img src="/assets/images/post/2025-03-19/multi-attach-efs.png" alt="Foo" />

  <figcaption>Mount EFS to multiple instances</figcaption>
</figure>

<h2 id="prerequisites">Prerequisites</h2>
<ul>
  <li>VPC, Three subnet (In here i will use public subnet), 1 IGW, 1 RTB</li>
  <li>3 Security Group. ALB SG (Allow HTTP, HTTPS), EC2 SG (Allow HTTP from ALB SG, SSH), EFS SG (Allow NFS from EC2 SG)</li>
  <li>Three EC2 Instance in different AZ (You can provision only two also)</li>
  <li>1 EFS Volume</li>
</ul>

<h2 id="terraform-implementation">Terraform Implementation</h2>

<h3 id="vpc-subnet-igw-and-rtb">VPC, Subnet IGW, and RTB</h3>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># Multi-Attaching Elastic File System (EFS) Volumes to EC2 with Application Load Balancers</span>
<span class="c1"># ====================================================================================================</span>

<span class="nx">provider</span> <span class="s2">"aws"</span> <span class="p">{</span>
  <span class="nx">region</span>  <span class="p">=</span> <span class="s2">"ap-southeast-1"</span>
  <span class="nx">profile</span> <span class="p">=</span> <span class="s2">"default"</span>

  <span class="nx">default_tags</span> <span class="p">{</span>
    <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
      <span class="nx">Owner</span>   <span class="p">=</span> <span class="s2">"nugroho-L1"</span>
      <span class="nx">Project</span> <span class="p">=</span> <span class="s2">"Testing"</span>
    <span class="p">}</span>
  <span class="p">}</span>
<span class="p">}</span>


<span class="c1"># ====================================================================================================</span>
<span class="c1"># VPC and Network Infra</span>
<span class="c1"># ====================================================================================================</span>

<span class="nx">resource</span> <span class="s2">"aws_vpc"</span> <span class="s2">"main"</span> <span class="p">{</span>
  <span class="nx">cidr_block</span>           <span class="p">=</span> <span class="s2">"192.168.0.0/16"</span>
  <span class="nx">instance_tenancy</span>     <span class="p">=</span> <span class="s2">"default"</span>
  <span class="nx">enable_dns_hostnames</span> <span class="p">=</span> <span class="kc">true</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"nugroho-vpc"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Subnets</span>
<span class="nx">resource</span> <span class="s2">"aws_subnet"</span> <span class="s2">"a"</span> <span class="p">{</span>
  <span class="nx">vpc_id</span>            <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_block</span>        <span class="p">=</span> <span class="s2">"192.168.1.0/24"</span>
  <span class="nx">availability_zone</span> <span class="p">=</span> <span class="s2">"ap-southeast-1a"</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"Public-Subnet-A-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_subnet"</span> <span class="s2">"b"</span> <span class="p">{</span>
  <span class="nx">vpc_id</span>            <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_block</span>        <span class="p">=</span> <span class="s2">"192.168.2.0/24"</span>
  <span class="nx">availability_zone</span> <span class="p">=</span> <span class="s2">"ap-southeast-1b"</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"Public-Subnet-B-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_subnet"</span> <span class="s2">"c"</span> <span class="p">{</span>
  <span class="nx">vpc_id</span>            <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_block</span>        <span class="p">=</span> <span class="s2">"192.168.3.0/24"</span>
  <span class="nx">availability_zone</span> <span class="p">=</span> <span class="s2">"ap-southeast-1c"</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"Public-Subnet-C-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Internet Gateway</span>
<span class="nx">resource</span> <span class="s2">"aws_internet_gateway"</span> <span class="s2">"gw"</span> <span class="p">{</span>
  <span class="nx">vpc_id</span> <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"main-igw-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Route Table</span>
<span class="nx">resource</span> <span class="s2">"aws_route_table"</span> <span class="s2">"main"</span> <span class="p">{</span>
  <span class="nx">vpc_id</span> <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>

  <span class="nx">route</span> <span class="p">{</span>
    <span class="nx">cidr_block</span> <span class="p">=</span> <span class="s2">"0.0.0.0/0"</span> <span class="c1"># Route to all external IPs</span>
    <span class="nx">gateway_id</span> <span class="p">=</span> <span class="nx">aws_internet_gateway</span><span class="err">.</span><span class="nx">gw</span><span class="err">.</span><span class="nx">id</span>
  <span class="p">}</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"main-route-table-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Route Table Associations</span>
<span class="nx">resource</span> <span class="s2">"aws_route_table_association"</span> <span class="s2">"a"</span> <span class="p">{</span>
  <span class="nx">subnet_id</span>      <span class="p">=</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">a</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">route_table_id</span> <span class="p">=</span> <span class="nx">aws_route_table</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_route_table_association"</span> <span class="s2">"b"</span> <span class="p">{</span>
  <span class="nx">subnet_id</span>      <span class="p">=</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">b</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">route_table_id</span> <span class="p">=</span> <span class="nx">aws_route_table</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_route_table_association"</span> <span class="s2">"c"</span> <span class="p">{</span>
  <span class="nx">subnet_id</span>      <span class="p">=</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">c</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">route_table_id</span> <span class="p">=</span> <span class="nx">aws_route_table</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
<span class="p">}</span>

</code></pre></div></div>

<h3 id="security-group">Security Group</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># Security Groups</span>
<span class="c1"># ====================================================================================================</span>

<span class="c1"># Security Group for ALB</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group"</span> <span class="s2">"alb-sec-group"</span> <span class="p">{</span>
  <span class="nx">name</span>        <span class="p">=</span> <span class="s2">"allow_http_https"</span>
  <span class="nx">description</span> <span class="p">=</span> <span class="s2">"Allow HTTP and HTTPS inbound traffic from anywhere"</span>
  <span class="nx">vpc_id</span>      <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"alb-sec-group-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Allow inbound HTTP traffic (port 80) from anywhere</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group_rule"</span> <span class="s2">"allow_http_alb"</span> <span class="p">{</span>
  <span class="nx">type</span>              <span class="p">=</span> <span class="s2">"ingress"</span>
  <span class="nx">from_port</span>         <span class="p">=</span> <span class="mi">80</span>
  <span class="nx">to_port</span>           <span class="p">=</span> <span class="mi">80</span>
  <span class="nx">protocol</span>          <span class="p">=</span> <span class="s2">"tcp"</span>
  <span class="nx">security_group_id</span> <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">alb</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_blocks</span>       <span class="p">=</span> <span class="p">[</span><span class="s2">"0.0.0.0/0"</span><span class="p">]</span>
<span class="p">}</span>

<span class="c1"># Allow inbound HTTPS traffic (port 443) from anywhere</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group_rule"</span> <span class="s2">"allow_https_alb"</span> <span class="p">{</span>
  <span class="nx">type</span>              <span class="p">=</span> <span class="s2">"ingress"</span>
  <span class="nx">from_port</span>         <span class="p">=</span> <span class="mi">443</span>
  <span class="nx">to_port</span>           <span class="p">=</span> <span class="mi">443</span>
  <span class="nx">protocol</span>          <span class="p">=</span> <span class="s2">"tcp"</span>
  <span class="nx">security_group_id</span> <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">alb</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_blocks</span>       <span class="p">=</span> <span class="p">[</span><span class="s2">"0.0.0.0/0"</span><span class="p">]</span>
<span class="p">}</span>

<span class="c1"># Allow all outbound traffic</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group_rule"</span> <span class="s2">"allow_all_outbound_alb"</span> <span class="p">{</span>
  <span class="nx">type</span>              <span class="p">=</span> <span class="s2">"egress"</span>
  <span class="nx">from_port</span>         <span class="p">=</span> <span class="mi">0</span>
  <span class="nx">to_port</span>           <span class="p">=</span> <span class="mi">0</span>
  <span class="nx">protocol</span>          <span class="p">=</span> <span class="s2">"-1"</span>
  <span class="nx">security_group_id</span> <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">alb</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_blocks</span>       <span class="p">=</span> <span class="p">[</span><span class="s2">"0.0.0.0/0"</span><span class="p">]</span>
<span class="p">}</span>


<span class="c1"># Security Group for EC2 or Target Group</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group"</span> <span class="s2">"ec2-sec-group"</span> <span class="p">{</span>
  <span class="nx">name</span>        <span class="p">=</span> <span class="s2">"allow_from-alb"</span>
  <span class="nx">description</span> <span class="p">=</span> <span class="s2">"Allow HTTP traffic from ALB and SSH from specific IP"</span>
  <span class="nx">vpc_id</span>      <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"ec2-sec-group-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Allow inbound HTTP traffic (port 80) from ALB only</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group_rule"</span> <span class="s2">"allow_http_from_alb"</span> <span class="p">{</span>
  <span class="nx">type</span>                     <span class="p">=</span> <span class="s2">"ingress"</span>
  <span class="nx">from_port</span>                <span class="p">=</span> <span class="mi">80</span>
  <span class="nx">to_port</span>                  <span class="p">=</span> <span class="mi">80</span>
  <span class="nx">protocol</span>                 <span class="p">=</span> <span class="s2">"tcp"</span>
  <span class="nx">security_group_id</span>        <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">ec2</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">source_security_group_id</span> <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">alb</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
<span class="p">}</span>

<span class="c1"># Allow inbound SSH traffic (port 22) from a specific IP</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group_rule"</span> <span class="s2">"allow_ssh"</span> <span class="p">{</span>
  <span class="nx">type</span>              <span class="p">=</span> <span class="s2">"ingress"</span>
  <span class="nx">from_port</span>         <span class="p">=</span> <span class="mi">22</span>
  <span class="nx">to_port</span>           <span class="p">=</span> <span class="mi">22</span>
  <span class="nx">protocol</span>          <span class="p">=</span> <span class="s2">"tcp"</span>
  <span class="nx">security_group_id</span> <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">ec2</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_blocks</span>       <span class="p">=</span> <span class="p">[</span><span class="s2">"0.0.0.0/0"</span><span class="p">]</span> <span class="c1"># Replace with your actual public IP</span>
<span class="p">}</span>

<span class="c1"># Allow all outbound traffic</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group_rule"</span> <span class="s2">"allow_all_outbound_ec2"</span> <span class="p">{</span>
  <span class="nx">type</span>              <span class="p">=</span> <span class="s2">"egress"</span>
  <span class="nx">from_port</span>         <span class="p">=</span> <span class="mi">0</span>
  <span class="nx">to_port</span>           <span class="p">=</span> <span class="mi">0</span>
  <span class="nx">protocol</span>          <span class="p">=</span> <span class="s2">"-1"</span>
  <span class="nx">security_group_id</span> <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">ec2</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_blocks</span>       <span class="p">=</span> <span class="p">[</span><span class="s2">"0.0.0.0/0"</span><span class="p">]</span>
<span class="p">}</span>


<span class="c1"># Security Group for EFS</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group"</span> <span class="s2">"efs-sec-group"</span> <span class="p">{</span>
  <span class="nx">name</span>        <span class="p">=</span> <span class="s2">"allow_from-ec2"</span>
  <span class="nx">description</span> <span class="p">=</span> <span class="s2">"Allow TCP traffic on port 2049 from EC2 instances"</span>
  <span class="nx">vpc_id</span>      <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"efs-sec-group-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Allow inbound 2049 from EC2 security group only</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group_rule"</span> <span class="s2">"allow_nfs_from_ec2"</span> <span class="p">{</span>
  <span class="nx">type</span>                     <span class="p">=</span> <span class="s2">"ingress"</span>
  <span class="nx">from_port</span>                <span class="p">=</span> <span class="mi">2049</span>
  <span class="nx">to_port</span>                  <span class="p">=</span> <span class="mi">2049</span>
  <span class="nx">protocol</span>                 <span class="p">=</span> <span class="s2">"tcp"</span>
  <span class="nx">security_group_id</span>        <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">efs</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">source_security_group_id</span> <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">ec2</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
<span class="p">}</span>

<span class="c1"># Allow all outbound traffic</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group_rule"</span> <span class="s2">"allow_all_outbound_efs"</span> <span class="p">{</span>
  <span class="nx">type</span>              <span class="p">=</span> <span class="s2">"egress"</span>
  <span class="nx">from_port</span>         <span class="p">=</span> <span class="mi">0</span>
  <span class="nx">to_port</span>           <span class="p">=</span> <span class="mi">0</span>
  <span class="nx">protocol</span>          <span class="p">=</span> <span class="s2">"-1"</span>
  <span class="nx">security_group_id</span> <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">efs</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_blocks</span>       <span class="p">=</span> <span class="p">[</span><span class="s2">"0.0.0.0/0"</span><span class="p">]</span>
<span class="p">}</span>

</code></pre></div></div>

<h3 id="ec2-instances">EC2 Instances</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># Launch an EC2 Instance</span>
<span class="c1"># ====================================================================================================</span>

<span class="c1"># Get latest Ubuntu AMI</span>
<span class="nx">data</span> <span class="s2">"aws_ami"</span> <span class="s2">"ubuntu"</span> <span class="p">{</span>
  <span class="nx">most_recent</span> <span class="p">=</span> <span class="kc">true</span>
  <span class="nx">owners</span>      <span class="p">=</span> <span class="p">[</span><span class="s2">"099720109477"</span><span class="p">]</span> <span class="c1"># The official Ubuntu AMI owner ID, if the owner using amazon, then it can't connect.</span>

  <span class="nx">filter</span> <span class="p">{</span>
    <span class="nx">name</span>   <span class="p">=</span> <span class="s2">"name"</span>
    <span class="nx">values</span> <span class="p">=</span> <span class="p">[</span><span class="s2">"ubuntu/images/hvm-ssd/ubuntu-*-amd64-server-*"</span><span class="p">]</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Define a map of subnets and availability zones dynamically using locals</span>
<span class="nx">locals</span> <span class="p">{</span>
  <span class="nx">subnets</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">a</span> <span class="p">=</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">a</span><span class="err">.</span><span class="nx">id</span>
    <span class="nx">b</span> <span class="p">=</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">b</span><span class="err">.</span><span class="nx">id</span>
    <span class="nx">c</span> <span class="p">=</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">c</span><span class="err">.</span><span class="nx">id</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Create the Ubuntu EC2 Web server</span>
<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">depends_on</span>                  <span class="p">=</span> <span class="p">[</span><span class="nx">aws_efs_file_system</span><span class="err">.</span><span class="nx">efs_file_system</span><span class="p">]</span>
  <span class="nx">for_each</span>                    <span class="p">=</span> <span class="nx">local</span><span class="err">.</span><span class="nx">subnets</span>
  <span class="nx">ami</span>                         <span class="p">=</span> <span class="nx">data</span><span class="err">.</span><span class="nx">aws_ami</span><span class="err">.</span><span class="nx">ubuntu</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">instance_type</span>               <span class="p">=</span> <span class="s2">"t3.micro"</span>
  <span class="nx">key_name</span>                    <span class="p">=</span> <span class="s2">"nugroho-msikp"</span>
  <span class="nx">subnet_id</span>                   <span class="p">=</span> <span class="nx">each</span><span class="err">.</span><span class="nx">value</span>
  <span class="nx">associate_public_ip_address</span> <span class="p">=</span> <span class="kc">true</span>
  <span class="nx">vpc_security_group_ids</span>      <span class="p">=</span> <span class="p">[</span><span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">ec2</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span><span class="p">]</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"web-server-apache-${each.key}-nugroho"</span>
  <span class="p">}</span>

  <span class="nx">user_data</span> <span class="p">=</span> <span class="o">&lt;&lt;-</span><span class="no">EOF</span><span class="sh">
  #!/bin/bash
  sudo apt-get update -y
  sudo apt-get install apache2 -y
  sudo systemctl start apache2
  sudo systemctl enable apache2

  sudo apt-get install -y amazon-efs-utils

  # Create the directory for EFS mount
  sudo mkdir /efs

  # Mount the EFS
  sudo mount -t efs -o tls ${aws_efs_file_system.efs_file_system.id}:/ /efs

  # Add EFS entry to /etc/fstab for auto-mount on reboot
  echo "${aws_efs_file_system.efs_file_system.id}:/ /mnt/efs efs _netdev,tls 0 0" | sudo tee -a /etc/fstab

  # Get EC2 metadata
  instanceId=$(curl http://169.254.169.254/latest/meta-data/instance-id)
  instanceAZ=$(curl http://169.254.169.254/latest/meta-data/placement/availability-zone)
  pubHostName=$(curl http://169.254.169.254/latest/meta-data/public-hostname)
  pubIPv4=$(curl http://169.254.169.254/latest/meta-data/public-ipv4)
  privHostName=$(curl http://169.254.169.254/latest/meta-data/local-hostname)
  privIPv4=$(curl http://169.254.169.254/latest/meta-data/local-ipv4)

  # Output instance metadata and EFS information
  echo "&lt;font face = 'Verdana' size = '5'&gt;"                               &gt; /var/www/html/index.html
  echo "&lt;center&gt;&lt;h1&gt;AWS Ubuntu VM Deployed with Terraform&lt;/h1&gt;&lt;/center&gt;"   &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;EC2 Instance Metadata&lt;/b&gt; &lt;/center&gt;"                  &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;Instance ID:&lt;/b&gt; $instanceId &lt;/center&gt;"                      &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;AWS Availability Zone:&lt;/b&gt; $instanceAZ &lt;/center&gt;"             &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;Public Hostname:&lt;/b&gt; $pubHostName &lt;/center&gt;"                 &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;Public IPv4:&lt;/b&gt; $pubIPv4 &lt;/center&gt;"                         &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;Private Hostname:&lt;/b&gt; $privHostName &lt;/center&gt;"               &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;Private IPv4:&lt;/b&gt; $privIPv4 &lt;/center&gt;"                       &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt;&lt;b&gt;EFS File System ID:&lt;/b&gt; ${aws_efs_file_system.efs_file_system.id}&lt;/center&gt;" &gt;&gt; /var/www/html/index.html
  echo "&lt;/font&gt;"                                                          &gt;&gt; /var/www/html/index.html
</span><span class="no">  EOF


</span><span class="p">}</span>


</code></pre></div></div>

<h3 id="application-load-balancer-and-target-group">Application Load Balancer and Target Group</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># Application Load Balancer</span>
<span class="c1"># ====================================================================================================</span>
<span class="nx">resource</span> <span class="s2">"aws_lb"</span> <span class="s2">"aws-application_load_balancer"</span> <span class="p">{</span>
  <span class="nx">name</span>                       <span class="p">=</span> <span class="s2">"app-load-balancer-nugroho"</span>
  <span class="nx">internal</span>                   <span class="p">=</span> <span class="kc">false</span>
  <span class="nx">load_balancer_type</span>         <span class="p">=</span> <span class="s2">"application"</span>
  <span class="nx">security_groups</span>            <span class="p">=</span> <span class="p">[</span><span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">alb</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span><span class="p">]</span>
  <span class="nx">subnets</span>                    <span class="p">=</span> <span class="nx">values</span><span class="err">(</span><span class="nx">local</span><span class="err">.</span><span class="nx">subnets</span><span class="err">)</span> <span class="c1"># All subnets</span>
  <span class="nx">enable_deletion_protection</span> <span class="p">=</span> <span class="kc">false</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"Application-Load-Balancer-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>


<span class="c1"># ====================================================================================================</span>
<span class="c1"># Target Group</span>
<span class="c1"># ====================================================================================================</span>

<span class="nx">resource</span> <span class="s2">"aws_lb_target_group"</span> <span class="s2">"app_tg"</span> <span class="p">{</span>
  <span class="nx">name</span>        <span class="p">=</span> <span class="s2">"app-target-group-nugroho"</span>
  <span class="nx">port</span>        <span class="p">=</span> <span class="mi">80</span>     <span class="c1"># Target Group port</span>
  <span class="nx">protocol</span>    <span class="p">=</span> <span class="s2">"HTTP"</span> <span class="c1"># Protocol to communicate with targets</span>
  <span class="nx">vpc_id</span>      <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">target_type</span> <span class="p">=</span> <span class="s2">"instance"</span> <span class="c1"># Target type (can be "ip" or "lambda" as well)</span>

  <span class="nx">health_check</span> <span class="p">{</span>
    <span class="nx">enabled</span>             <span class="p">=</span> <span class="kc">true</span>
    <span class="nx">interval</span>            <span class="p">=</span> <span class="mi">30</span> <span class="c1"># Check every 30 seconds</span>
    <span class="nx">path</span>                <span class="p">=</span> <span class="s2">"/"</span>
    <span class="nx">timeout</span>             <span class="p">=</span> <span class="mi">5</span>
    <span class="nx">matcher</span>             <span class="p">=</span> <span class="mi">200</span>
    <span class="nx">healthy_threshold</span>   <span class="p">=</span> <span class="mi">5</span> <span class="c1"># Mark healthy after 2 successes</span>
    <span class="nx">unhealthy_threshold</span> <span class="p">=</span> <span class="mi">5</span> <span class="c1"># Mark unhealthy after 2 failures</span>
  <span class="p">}</span>
  <span class="nx">lifecycle</span> <span class="p">{</span>
    <span class="nx">create_before_destroy</span> <span class="p">=</span> <span class="kc">true</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Attach EC2 instances to the Target Group</span>
<span class="nx">resource</span> <span class="s2">"aws_lb_target_group_attachment"</span> <span class="s2">"app_tg_attachment"</span> <span class="p">{</span>
  <span class="nx">for_each</span> <span class="p">=</span> <span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web</span> <span class="c1"># Loop over all instances</span>

  <span class="nx">target_group_arn</span> <span class="p">=</span> <span class="nx">aws_lb_target_group</span><span class="err">.</span><span class="nx">app_tg</span><span class="err">.</span><span class="nx">arn</span>
  <span class="nx">target_id</span>        <span class="p">=</span> <span class="nx">each</span><span class="err">.</span><span class="nx">value</span><span class="err">.</span><span class="nx">id</span> <span class="c1"># Attach each instance by ID</span>
  <span class="nx">port</span>             <span class="p">=</span> <span class="mi">80</span>            <span class="c1"># Port for the target</span>
<span class="p">}</span>


<span class="c1"># ====================================================================================================</span>
<span class="c1"># Listener</span>
<span class="c1"># ====================================================================================================</span>

<span class="nx">resource</span> <span class="s2">"aws_lb_listener"</span> <span class="s2">"http_listener"</span> <span class="p">{</span>
  <span class="nx">load_balancer_arn</span> <span class="p">=</span> <span class="nx">aws_lb</span><span class="err">.</span><span class="nx">aws</span><span class="err">-</span><span class="nx">application_load_balancer</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">port</span>              <span class="p">=</span> <span class="mi">80</span> <span class="c1"># Listener port</span>
  <span class="nx">protocol</span>          <span class="p">=</span> <span class="s2">"HTTP"</span>

  <span class="nx">default_action</span> <span class="p">{</span>
    <span class="nx">type</span>             <span class="p">=</span> <span class="s2">"forward"</span>
    <span class="nx">target_group_arn</span> <span class="p">=</span> <span class="nx">aws_lb_target_group</span><span class="err">.</span><span class="nx">app_tg</span><span class="err">.</span><span class="nx">arn</span>
  <span class="p">}</span>
<span class="p">}</span>

</code></pre></div></div>

<h3 id="elastic-file-system">Elastic File System</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># Configuring Elastic File System (EFS)</span>
<span class="c1"># ====================================================================================================</span>

<span class="c1"># Create EFS</span>
<span class="nx">resource</span> <span class="s2">"aws_efs_file_system"</span> <span class="s2">"efs_file_system"</span> <span class="p">{</span>
  <span class="nx">creation_token</span>   <span class="p">=</span> <span class="s2">"efs-test"</span>
  <span class="nx">performance_mode</span> <span class="p">=</span> <span class="s2">"generalPurpose"</span>
  <span class="nx">throughput_mode</span>  <span class="p">=</span> <span class="s2">"bursting"</span>
  <span class="nx">lifecycle_policy</span> <span class="p">{</span>
    <span class="nx">transition_to_ia</span> <span class="p">=</span> <span class="s2">"AFTER_30_DAYS"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Create EFS Mount Targets for each subnet</span>
<span class="nx">resource</span> <span class="s2">"aws_efs_mount_target"</span> <span class="s2">"mount_targets"</span> <span class="p">{</span>
  <span class="nx">count</span>           <span class="p">=</span> <span class="mi">3</span>
  <span class="nx">file_system_id</span>  <span class="p">=</span> <span class="nx">aws_efs_file_system</span><span class="err">.</span><span class="nx">efs_file_system</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">subnet_id</span>       <span class="p">=</span> <span class="p">[</span><span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">a</span><span class="err">.</span><span class="nx">id</span><span class="p">,</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">b</span><span class="err">.</span><span class="nx">id</span><span class="p">,</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">c</span><span class="err">.</span><span class="nx">id</span><span class="p">][</span><span class="nx">count</span><span class="err">.</span><span class="nx">index</span><span class="p">]</span> <span class="c1"># Mount target per subnet</span>
  <span class="nx">security_groups</span> <span class="p">=</span> <span class="p">[</span><span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">efs</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span><span class="p">]</span>
<span class="p">}</span>

</code></pre></div></div>

<h3 id="outputs">Outputs</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># Outputs</span>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># Output the DNS of ALB</span>
<span class="nx">output</span> <span class="s2">"alb_dns_name"</span> <span class="p">{</span>
  <span class="nx">value</span>       <span class="p">=</span> <span class="nx">aws_lb</span><span class="err">.</span><span class="nx">aws</span><span class="err">-</span><span class="nx">application_load_balancer</span><span class="err">.</span><span class="nx">dns_name</span>
  <span class="nx">description</span> <span class="p">=</span> <span class="s2">"DNS name of the Application Load Balancer"</span>
<span class="p">}</span>

<span class="c1"># Output public IPs of all EC2 instances</span>
<span class="nx">output</span> <span class="s2">"instance_public_ips"</span> <span class="p">{</span>
  <span class="nx">value</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">for</span> <span class="nx">key</span><span class="err">,</span> <span class="nx">instance</span> <span class="nx">in</span> <span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web</span> <span class="err">:</span> <span class="nx">key</span> <span class="p">=</span><span class="err">&gt;</span> <span class="nx">instance</span><span class="err">.</span><span class="nx">public_ip</span>
  <span class="p">}</span>
  <span class="nx">description</span> <span class="p">=</span> <span class="s2">"Public IPs of the EC2 instances"</span>
<span class="p">}</span>

<span class="c1"># Output private IPs of all EC2 instances</span>
<span class="nx">output</span> <span class="s2">"instance_private_ips"</span> <span class="p">{</span>
  <span class="nx">value</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">for</span> <span class="nx">key</span><span class="err">,</span> <span class="nx">instance</span> <span class="nx">in</span> <span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web</span> <span class="err">:</span> <span class="nx">key</span> <span class="p">=</span><span class="err">&gt;</span> <span class="nx">instance</span><span class="err">.</span><span class="nx">private_ip</span>
  <span class="p">}</span>
  <span class="nx">description</span> <span class="p">=</span> <span class="s2">"Private IPs of the EC2 instances"</span>
<span class="p">}</span>

</code></pre></div></div>

<h2 id="test-create-file">Test Create File</h2>

<p>If you access the EC2 and create a file in one of the EC2. And then, try acessing the file in another ec2, you’ll find the same file.</p>

<h2 id="common-errors-and-troubleshooting">Common Errors and Troubleshooting</h2>

<h3 id="cant-access-the-browser">Can’t access the browser</h3>
<p>If you can’t access the instance in the browser, please check the security group that you use, make sure that the security group allow inbound HTTP/HTTPS from anywhere.</p>

<h3 id="attachmentlimitexceeded-error">AttachmentLimitExceeded error</h3>
<p>Btw there are number of limit when attach ebs volume to the instance, it depends on the instance type.</p>

<p><strong>References:</strong></p>
<ul>
  <li>https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/volume_limits.html</li>
</ul>]]></content><author><name>Nugroho</name></author><category term="cloud" /><category term="aws" /><category term="efs" /><summary type="html"><![CDATA[Amazon Elastic File System (EFS) is a fully managed, scalable file storage service designed to be shared across multiple compute resources including Amazon EC2, Amazon ECS, Amazon EKS, AWS Lambda, and AWS Fargate. Unlike Amazon EBS, which is typically attached to a single instance (with limited multi-attach support), EFS is built from the ground up to support multiple EC2 instances mounting the same filesystem simultaneously.]]></summary></entry><entry><title type="html">Multi Attach EBS</title><link href="https://galnug.com/multi-attach-ebs/" rel="alternate" type="text/html" title="Multi Attach EBS" /><published>2025-02-12T21:00:00+07:00</published><updated>2025-02-12T21:00:00+07:00</updated><id>https://galnug.com/multi-attach-ebs</id><content type="html" xml:base="https://galnug.com/multi-attach-ebs/"><![CDATA[<p>As we know that the EBS Volume can’t be attached to instance that are in different AZ. But, the EBS volume can be attached to multiple instance in the same  AZ. If you read <a href="https://docs.aws.amazon.com/ebs/latest/userguide/ebs-volumes-multi.html">in this documentation</a>, EBS only can be muliti attach only for volume type <code class="language-plaintext highlighter-rouge">io1</code> and <code class="language-plaintext highlighter-rouge">io2</code>, it can’t be done using another volume type.</p>

<h2 id="architecture-overview">Architecture Overview</h2>
<p>The following architecture is used to attach EBS Volume to multiple instance.</p>

<figure>
  
<img src="/assets/images/post/2025-02-12/multi-attach-ebs.png" alt="Foo" />

  <figcaption>Multi Attach EBS Volume</figcaption>
</figure>

<h2 id="prerequisites">Prerequisites</h2>
<ul>
  <li>VPC, Three subnet (In here i will use public subnet), 1 IGW, 1 RTB</li>
  <li>1 Security Group (Allow SSH, HTTP, HTTPS)</li>
  <li>Three EC2 Instance in the same AZ (You can provision only two also)</li>
  <li>1 EBS Volume io2</li>
</ul>

<h2 id="terraform-implementation">Terraform Implementation</h2>

<h3 id="vpc-subnet-igw-and-rtb">VPC, Subnet IGW, and RTB</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># Configuring Multi-Attach EBS Volumes with Terraform</span>
<span class="c1"># ====================================================================================================</span>

<span class="nx">provider</span> <span class="s2">"aws"</span> <span class="p">{</span>
  <span class="nx">region</span>  <span class="p">=</span> <span class="s2">"ap-southeast-1"</span>
  <span class="nx">profile</span> <span class="p">=</span> <span class="s2">"default"</span>

  <span class="nx">default_tags</span> <span class="p">{</span>
    <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
      <span class="nx">Owner</span>   <span class="p">=</span> <span class="s2">"nugroho-L1"</span>
      <span class="nx">Project</span> <span class="p">=</span> <span class="s2">"Testing"</span>
    <span class="p">}</span>
  <span class="p">}</span>
<span class="p">}</span>


<span class="c1"># ====================================================================================================</span>
<span class="c1"># VPC and Network Infra</span>
<span class="c1"># ====================================================================================================</span>

<span class="nx">resource</span> <span class="s2">"aws_vpc"</span> <span class="s2">"main"</span> <span class="p">{</span>
  <span class="nx">cidr_block</span>           <span class="p">=</span> <span class="s2">"192.168.0.0/16"</span>
  <span class="nx">instance_tenancy</span>     <span class="p">=</span> <span class="s2">"default"</span>
  <span class="nx">enable_dns_hostnames</span> <span class="p">=</span> <span class="kc">true</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"nugroho-vpc"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Subnets</span>
<span class="nx">resource</span> <span class="s2">"aws_subnet"</span> <span class="s2">"a"</span> <span class="p">{</span>
  <span class="nx">vpc_id</span>            <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_block</span>        <span class="p">=</span> <span class="s2">"192.168.1.0/24"</span>
  <span class="nx">availability_zone</span> <span class="p">=</span> <span class="s2">"ap-southeast-1a"</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"Public-Subnet-A-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_subnet"</span> <span class="s2">"b"</span> <span class="p">{</span>
  <span class="nx">vpc_id</span>            <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_block</span>        <span class="p">=</span> <span class="s2">"192.168.2.0/24"</span>
  <span class="nx">availability_zone</span> <span class="p">=</span> <span class="s2">"ap-southeast-1a"</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"Public-Subnet-B-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_subnet"</span> <span class="s2">"c"</span> <span class="p">{</span>
  <span class="nx">vpc_id</span>            <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_block</span>        <span class="p">=</span> <span class="s2">"192.168.3.0/24"</span>
  <span class="nx">availability_zone</span> <span class="p">=</span> <span class="s2">"ap-southeast-1a"</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"Public-Subnet-C-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Internet Gateway</span>
<span class="nx">resource</span> <span class="s2">"aws_internet_gateway"</span> <span class="s2">"gw"</span> <span class="p">{</span>
  <span class="nx">vpc_id</span> <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"main-igw-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Route Table</span>
<span class="nx">resource</span> <span class="s2">"aws_route_table"</span> <span class="s2">"main"</span> <span class="p">{</span>
  <span class="nx">vpc_id</span> <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>

  <span class="nx">route</span> <span class="p">{</span>
    <span class="nx">cidr_block</span> <span class="p">=</span> <span class="s2">"0.0.0.0/0"</span> <span class="c1"># Route to all external IPs</span>
    <span class="nx">gateway_id</span> <span class="p">=</span> <span class="nx">aws_internet_gateway</span><span class="err">.</span><span class="nx">gw</span><span class="err">.</span><span class="nx">id</span>
  <span class="p">}</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"main-route-table-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Route Table Associations</span>
<span class="nx">resource</span> <span class="s2">"aws_route_table_association"</span> <span class="s2">"a"</span> <span class="p">{</span>
  <span class="nx">subnet_id</span>      <span class="p">=</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">a</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">route_table_id</span> <span class="p">=</span> <span class="nx">aws_route_table</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_route_table_association"</span> <span class="s2">"b"</span> <span class="p">{</span>
  <span class="nx">subnet_id</span>      <span class="p">=</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">b</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">route_table_id</span> <span class="p">=</span> <span class="nx">aws_route_table</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_route_table_association"</span> <span class="s2">"c"</span> <span class="p">{</span>
  <span class="nx">subnet_id</span>      <span class="p">=</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">c</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">route_table_id</span> <span class="p">=</span> <span class="nx">aws_route_table</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>
<span class="p">}</span>

</code></pre></div></div>

<h3 id="security-group">Security Group</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># Security Groups</span>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># Security Group for EC2</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group"</span> <span class="s2">"ec2-sec-group"</span> <span class="p">{</span>
  <span class="nx">name</span>        <span class="p">=</span> <span class="s2">"allow_from-ssh"</span>
  <span class="nx">description</span> <span class="p">=</span> <span class="s2">"Allow ssh and http and https from anywhere"</span>
  <span class="nx">vpc_id</span>      <span class="p">=</span> <span class="nx">aws_vpc</span><span class="err">.</span><span class="nx">main</span><span class="err">.</span><span class="nx">id</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"ec2-sec-group-nugroho"</span>
  <span class="p">}</span>
<span class="p">}</span>


<span class="c1"># Allow inbound SSH traffic (port 22) from anywhere</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group_rule"</span> <span class="s2">"allow_ssh"</span> <span class="p">{</span>
  <span class="nx">type</span>              <span class="p">=</span> <span class="s2">"ingress"</span>
  <span class="nx">from_port</span>         <span class="p">=</span> <span class="mi">22</span>
  <span class="nx">to_port</span>           <span class="p">=</span> <span class="mi">22</span>
  <span class="nx">protocol</span>          <span class="p">=</span> <span class="s2">"tcp"</span>
  <span class="nx">security_group_id</span> <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">ec2</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_blocks</span>       <span class="p">=</span> <span class="p">[</span><span class="s2">"0.0.0.0/0"</span><span class="p">]</span>
<span class="p">}</span>

<span class="c1"># Allow inbound HTTP traffic (port 80) from anywhere</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group_rule"</span> <span class="s2">"allow_http"</span> <span class="p">{</span>
  <span class="nx">type</span>              <span class="p">=</span> <span class="s2">"ingress"</span>
  <span class="nx">from_port</span>         <span class="p">=</span> <span class="mi">80</span>
  <span class="nx">to_port</span>           <span class="p">=</span> <span class="mi">80</span>
  <span class="nx">protocol</span>          <span class="p">=</span> <span class="s2">"tcp"</span>
  <span class="nx">security_group_id</span> <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">ec2</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_blocks</span>       <span class="p">=</span> <span class="p">[</span><span class="s2">"0.0.0.0/0"</span><span class="p">]</span>
<span class="p">}</span>

<span class="c1"># Allow inbound HTTPS traffic (port 443) from anywhere</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group_rule"</span> <span class="s2">"allow_https"</span> <span class="p">{</span>
  <span class="nx">type</span>              <span class="p">=</span> <span class="s2">"ingress"</span>
  <span class="nx">from_port</span>         <span class="p">=</span> <span class="mi">443</span>
  <span class="nx">to_port</span>           <span class="p">=</span> <span class="mi">443</span>
  <span class="nx">protocol</span>          <span class="p">=</span> <span class="s2">"tcp"</span>
  <span class="nx">security_group_id</span> <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">ec2</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_blocks</span>       <span class="p">=</span> <span class="p">[</span><span class="s2">"0.0.0.0/0"</span><span class="p">]</span>
<span class="p">}</span>


<span class="c1"># Allow all outbound traffic</span>
<span class="nx">resource</span> <span class="s2">"aws_security_group_rule"</span> <span class="s2">"allow_all_outbound_ec2"</span> <span class="p">{</span>
  <span class="nx">type</span>              <span class="p">=</span> <span class="s2">"egress"</span>
  <span class="nx">from_port</span>         <span class="p">=</span> <span class="mi">0</span>
  <span class="nx">to_port</span>           <span class="p">=</span> <span class="mi">0</span>
  <span class="nx">protocol</span>          <span class="p">=</span> <span class="s2">"-1"</span>
  <span class="nx">security_group_id</span> <span class="p">=</span> <span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">ec2</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">cidr_blocks</span>       <span class="p">=</span> <span class="p">[</span><span class="s2">"0.0.0.0/0"</span><span class="p">]</span>
<span class="p">}</span>


</code></pre></div></div>

<h3 id="ec2-instance">EC2 Instance</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># Launch an EC2 Instance</span>
<span class="c1"># ====================================================================================================</span>

<span class="c1"># Get latest Ubuntu AMI</span>
<span class="nx">data</span> <span class="s2">"aws_ami"</span> <span class="s2">"ubuntu"</span> <span class="p">{</span>
  <span class="nx">most_recent</span> <span class="p">=</span> <span class="kc">true</span>
  <span class="nx">owners</span>      <span class="p">=</span> <span class="p">[</span><span class="s2">"099720109477"</span><span class="p">]</span> <span class="c1"># The official Ubuntu AMI owner ID, if the owner using amazon, then it can't connect.</span>

  <span class="nx">filter</span> <span class="p">{</span>
    <span class="nx">name</span>   <span class="p">=</span> <span class="s2">"name"</span>
    <span class="nx">values</span> <span class="p">=</span> <span class="p">[</span><span class="s2">"ubuntu/images/hvm-ssd/ubuntu-*-amd64-server-*"</span><span class="p">]</span>
  <span class="p">}</span>
<span class="p">}</span>


<span class="c1"># Define a map of subnets and availability zones dynamically using locals</span>
<span class="nx">locals</span> <span class="p">{</span>
  <span class="nx">subnets</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">a</span> <span class="p">=</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">a</span><span class="err">.</span><span class="nx">id</span>
    <span class="nx">b</span> <span class="p">=</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">b</span><span class="err">.</span><span class="nx">id</span>
    <span class="nx">c</span> <span class="p">=</span> <span class="nx">aws_subnet</span><span class="err">.</span><span class="nx">c</span><span class="err">.</span><span class="nx">id</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># Create the Ubuntu EC2 Web server</span>
<span class="nx">resource</span> <span class="s2">"aws_instance"</span> <span class="s2">"web"</span> <span class="p">{</span>
  <span class="nx">for_each</span>                    <span class="p">=</span> <span class="nx">local</span><span class="err">.</span><span class="nx">subnets</span>
  <span class="nx">ami</span>                         <span class="p">=</span> <span class="nx">data</span><span class="err">.</span><span class="nx">aws_ami</span><span class="err">.</span><span class="nx">ubuntu</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">instance_type</span>               <span class="p">=</span> <span class="s2">"t3.micro"</span>
  <span class="nx">key_name</span>                    <span class="p">=</span> <span class="s2">"nugroho-msikp"</span>
  <span class="nx">subnet_id</span>                   <span class="p">=</span> <span class="nx">each</span><span class="err">.</span><span class="nx">value</span>
  <span class="nx">associate_public_ip_address</span> <span class="p">=</span> <span class="kc">true</span>
  <span class="nx">vpc_security_group_ids</span>      <span class="p">=</span> <span class="p">[</span><span class="nx">aws_security_group</span><span class="err">.</span><span class="nx">ec2</span><span class="err">-</span><span class="nx">sec</span><span class="err">-</span><span class="nx">group</span><span class="err">.</span><span class="nx">id</span><span class="p">]</span>

  <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"web-server-apache-${each.key}-nugroho"</span>
  <span class="p">}</span>

  <span class="nx">user_data</span> <span class="p">=</span> <span class="o">&lt;&lt;-</span><span class="no">EOF</span><span class="sh">
  #!/bin/bash
  sudo apt-get update -y
  sudo apt-get install apache2 -y
  sudo systemctl start apache2
  sudo systemctl enable apache2
  instanceId=$(curl http://169.254.169.254/latest/meta-data/instance-id)
  instanceAZ=$(curl http://169.254.169.254/latest/meta-data/placement/availability-zone)
  pubHostName=$(curl http://169.254.169.254/latest/meta-data/public-hostname)
  pubIPv4=$(curl http://169.254.169.254/latest/meta-data/public-ipv4)
  privHostName=$(curl http://169.254.169.254/latest/meta-data/local-hostname)
  privIPv4=$(curl http://169.254.169.254/latest/meta-data/local-ipv4)

  echo "&lt;font face = 'Verdana' size = '5'&gt;"                               &gt; /var/www/html/index.html
  echo "&lt;center&gt;&lt;h1&gt;AWS Ubuntu VM Deployed with Terraform&lt;/h1&gt;&lt;/center&gt;"   &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;EC2 Instance Metadata&lt;/b&gt; &lt;/center&gt;"                  &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;Instance ID:&lt;/b&gt; $instanceId &lt;/center&gt;"                      &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;AWS Availability Zone:&lt;/b&gt; $instanceAZ &lt;/center&gt;"             &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;Public Hostname:&lt;/b&gt; $pubHostName &lt;/center&gt;"                 &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;Public IPv4:&lt;/b&gt; $pubIPv4 &lt;/center&gt;"                         &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;Private Hostname:&lt;/b&gt; $privHostName &lt;/center&gt;"               &gt;&gt; /var/www/html/index.html
  echo "&lt;center&gt; &lt;b&gt;Private IPv4:&lt;/b&gt; $privIPv4 &lt;/center&gt;"                       &gt;&gt; /var/www/html/index.html
  echo "&lt;/font&gt;"                                                          &gt;&gt; /var/www/html/index.html
</span><span class="no">  EOF
</span><span class="p">}</span>

</code></pre></div></div>

<h3 id="ebs-volume-and-attachment">EBS Volume and Attachment</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># EBS Multi Attach Volume and Attach to all EC2 Instances</span>
<span class="c1"># ====================================================================================================</span>

<span class="nx">resource</span> <span class="s2">"aws_ebs_volume"</span> <span class="s2">"volume"</span> <span class="p">{</span>
  <span class="nx">availability_zone</span>    <span class="p">=</span> <span class="s2">"ap-southeast-1a"</span>
  <span class="nx">size</span>                 <span class="p">=</span> <span class="mi">4</span>
  <span class="nx">type</span>                 <span class="p">=</span> <span class="s2">"io2"</span>
  <span class="nx">iops</span>                 <span class="p">=</span> <span class="mi">200</span>
  <span class="nx">multi_attach_enabled</span> <span class="p">=</span> <span class="kc">true</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_volume_attachment"</span> <span class="s2">"ebs_attach"</span> <span class="p">{</span>
  <span class="nx">for_each</span>    <span class="p">=</span> <span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web</span>
  <span class="nx">device_name</span> <span class="p">=</span> <span class="s2">"/dev/sdh"</span>
  <span class="nx">volume_id</span>   <span class="p">=</span> <span class="nx">aws_ebs_volume</span><span class="err">.</span><span class="nx">volume</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">instance_id</span> <span class="p">=</span> <span class="nx">each</span><span class="err">.</span><span class="nx">value</span><span class="err">.</span><span class="nx">id</span>
<span class="p">}</span>

</code></pre></div></div>

<h3 id="outputs">Outputs</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># ====================================================================================================</span>
<span class="c1"># Outputs</span>
<span class="c1"># ====================================================================================================</span>

<span class="c1"># Output public IPs of all EC2 instances</span>
<span class="nx">output</span> <span class="s2">"instance_public_ips"</span> <span class="p">{</span>
  <span class="nx">value</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">for</span> <span class="nx">key</span><span class="err">,</span> <span class="nx">instance</span> <span class="nx">in</span> <span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web</span> <span class="err">:</span> <span class="nx">key</span> <span class="p">=</span><span class="err">&gt;</span> <span class="nx">instance</span><span class="err">.</span><span class="nx">public_ip</span>
  <span class="p">}</span>
  <span class="nx">description</span> <span class="p">=</span> <span class="s2">"Public IPs of the EC2 instances"</span>
<span class="p">}</span>

<span class="c1"># Output private IPs of all EC2 instances</span>
<span class="nx">output</span> <span class="s2">"instance_private_ips"</span> <span class="p">{</span>
  <span class="nx">value</span> <span class="p">=</span> <span class="p">{</span>
    <span class="nx">for</span> <span class="nx">key</span><span class="err">,</span> <span class="nx">instance</span> <span class="nx">in</span> <span class="nx">aws_instance</span><span class="err">.</span><span class="nx">web</span> <span class="err">:</span> <span class="nx">key</span> <span class="p">=</span><span class="err">&gt;</span> <span class="nx">instance</span><span class="err">.</span><span class="nx">private_ip</span>
  <span class="p">}</span>
  <span class="nx">description</span> <span class="p">=</span> <span class="s2">"Private IPs of the EC2 instances"</span>
<span class="p">}</span>

</code></pre></div></div>

<h2 id="access-the-vmec2">Access the VM/EC2</h2>

<p>I’ve already outputs the instance public IP in the terraform script, if you access the instance in the browser, and try refresh the browser several times, you’ll see that the instance we access is differents.</p>

<h2 id="common-errors-and-troubleshooting">Common Errors and Troubleshooting</h2>

<h3 id="cant-access-the-browser">Can’t access the browser</h3>
<p>If you can’t access the instance in the browser, please check the security group that you use, make sure that the security group allow inbound HTTP/HTTPS from anywhere.</p>

<h3 id="attachmentlimitexceeded-error">AttachmentLimitExceeded error</h3>
<p>Btw there are number of limit when attaching ebs volume to the instance, it depends on the instance type.</p>

<p><strong>References:</strong></p>
<ul>
  <li>https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/volume_limits.html</li>
</ul>]]></content><author><name>Nugroho</name></author><category term="cloud" /><category term="aws" /><category term="ebs" /><summary type="html"><![CDATA[As we know that the EBS Volume can’t be attached to instance that are in different AZ. But, the EBS volume can be attached to multiple instance in the same AZ. If you read in this documentation, EBS only can be muliti attach only for volume type io1 and io2, it can’t be done using another volume type.]]></summary></entry><entry><title type="html">Move Data From S3 to S3 Using Datasync</title><link href="https://galnug.com/move-data-from-s3-to-s3-using-datasync/" rel="alternate" type="text/html" title="Move Data From S3 to S3 Using Datasync" /><published>2025-01-15T21:00:00+07:00</published><updated>2025-01-15T21:00:00+07:00</updated><id>https://galnug.com/move-data-from-s3-to-s3-using-datasync</id><content type="html" xml:base="https://galnug.com/move-data-from-s3-to-s3-using-datasync/"><![CDATA[<p>AWS DataSync is a managed data transfer service that makes it easy to move data between on-premises storage and AWS storage services, as well as between AWS storage services themselves. In this guide, i’ll walk through the steps to move data from one S3 bucket to another S3 bucket using AWS DataSync.</p>

<h2 id="what-is-aws-datasync">What is AWS Datasync?</h2>
<p>AWS DataSync is a fully managed data transfer service that simplifies moving large amounts of data between on-premises storage and AWS services, or between AWS storage services such as Amazon S3, EFS, and FSx. In this guide, i’ll walk through the steps to move data from one S3 bucket to another S3 bucket using AWS DataSync.</p>

<h2 id="architecture-overview">Architecture Overview</h2>
<p>The following architecture is used to transfer data between two S3 buckets using AWS DataSync.</p>

<figure>
  
<img src="/assets/images/post/2025-01-15/s3-datasync-s3-same-region.png" alt="Foo" />

  <figcaption>Move CSV File from S3 to S3 using Datasync</figcaption>
</figure>

<h2 id="prerequisites">Prerequisites</h2>

<p>Before starting, make sure you have:</p>
<ul>
  <li>Two S3 buckets (source and destination)</li>
  <li>IAM permissions to create DataSync resources</li>
  <li>Terraform installed (v1.x or later)</li>
  <li>AWS CLI configured</li>
</ul>

<h2 id="terraform-implementation">Terraform Implementation</h2>

<h3 id="two-s3-bucket-source--destination">Two S3 Bucket (Source &amp; Destination)</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># provider configuration</span>
<span class="nx">provider</span> <span class="s2">"aws"</span> <span class="p">{</span>
  <span class="nx">region</span>  <span class="p">=</span> <span class="s2">"ap-southeast-1"</span>
  <span class="nx">profile</span> <span class="p">=</span> <span class="s2">"default"</span>
  <span class="nx">default_tags</span> <span class="p">{</span>
    <span class="nx">tags</span> <span class="p">=</span> <span class="p">{</span>
      <span class="nx">Owner</span>   <span class="p">=</span> <span class="s2">"Nugroho-L1"</span>
      <span class="nx">Project</span> <span class="p">=</span> <span class="s2">"Testing"</span>
    <span class="p">}</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># create source and destination s3 bucket</span>
<span class="nx">resource</span> <span class="s2">"aws_s3_bucket"</span> <span class="s2">"s3_source"</span> <span class="p">{</span>
  <span class="nx">bucket</span>        <span class="p">=</span> <span class="s2">"nugrohosource-testing-bucket"</span>
  <span class="nx">force_destroy</span> <span class="p">=</span> <span class="kc">true</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_s3_bucket"</span> <span class="s2">"s3_destination"</span> <span class="p">{</span>
  <span class="nx">bucket</span>        <span class="p">=</span> <span class="s2">"nugrohodestination-testing-bucket"</span>
  <span class="nx">force_destroy</span> <span class="p">=</span> <span class="kc">true</span>
<span class="p">}</span>

<span class="c1"># disable ACLs for s3 destination account</span>
<span class="nx">resource</span> <span class="s2">"aws_s3_bucket_ownership_controls"</span> <span class="s2">"s3_destination_disable_acls"</span> <span class="p">{</span>
  <span class="nx">bucket</span> <span class="p">=</span> <span class="nx">aws_s3_bucket</span><span class="err">.</span><span class="nx">s3_destination</span><span class="err">.</span><span class="nx">id</span>
  <span class="nx">rule</span> <span class="p">{</span>
    <span class="nx">object_ownership</span> <span class="p">=</span> <span class="s2">"BucketOwnerPreferred"</span>
  <span class="p">}</span>
<span class="p">}</span>

</code></pre></div></div>

<h3 id="iam-policy-for-s3-bucket">IAM Policy for S3 Bucket</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># create iam role for datasync (This Policy is created for the source bucket)</span>
<span class="nx">resource</span> <span class="s2">"aws_iam_role"</span> <span class="s2">"datasync_role"</span> <span class="p">{</span>
  <span class="nx">name</span> <span class="p">=</span> <span class="s2">"source-datasync-role"</span> <span class="c1"># the name of the role should be harcoded</span>

  <span class="nx">assume_role_policy</span> <span class="p">=</span> <span class="nx">jsonencode</span><span class="err">(</span><span class="p">{</span>
    <span class="nx">Version</span> <span class="p">=</span> <span class="s2">"2012-10-17"</span>
    <span class="nx">Statement</span> <span class="p">=</span> <span class="p">[</span>
      <span class="p">{</span>
        <span class="nx">Action</span> <span class="p">=</span> <span class="s2">"sts:AssumeRole"</span>
        <span class="nx">Effect</span> <span class="p">=</span> <span class="s2">"Allow"</span>
        <span class="nx">Principal</span> <span class="p">=</span> <span class="p">{</span>
          <span class="nx">Service</span> <span class="p">=</span> <span class="s2">"datasync.amazonaws.com"</span>
        <span class="p">}</span>
      <span class="p">}</span>
    <span class="p">]</span>
  <span class="p">}</span><span class="err">)</span>
<span class="p">}</span>

<span class="c1"># inline policy for the iam role</span>
<span class="nx">resource</span> <span class="s2">"aws_iam_role_policy"</span> <span class="s2">"datasync_policy"</span> <span class="p">{</span>
  <span class="nx">name</span> <span class="p">=</span> <span class="s2">"DataSyncS3Policy"</span>
  <span class="nx">role</span> <span class="p">=</span> <span class="nx">aws_iam_role</span><span class="err">.</span><span class="nx">datasync_role</span><span class="err">.</span><span class="nx">id</span>

  <span class="nx">policy</span> <span class="p">=</span> <span class="nx">jsonencode</span><span class="err">(</span><span class="p">{</span>
    <span class="nx">Version</span> <span class="p">=</span> <span class="s2">"2012-10-17"</span>
    <span class="nx">Statement</span> <span class="p">=</span> <span class="p">[</span>
      <span class="c1"># Permissions for the source bucket</span>
      <span class="p">{</span>
        <span class="nx">Action</span> <span class="p">=</span> <span class="p">[</span>
          <span class="s2">"s3:GetBucketLocation"</span><span class="p">,</span>
          <span class="s2">"s3:ListBucket"</span><span class="p">,</span>
          <span class="s2">"s3:ListBucketMultipartUploads"</span>
        <span class="p">]</span>
        <span class="nx">Effect</span>   <span class="p">=</span> <span class="s2">"Allow"</span>
        <span class="nx">Resource</span> <span class="p">=</span> <span class="nx">aws_s3_bucket</span><span class="err">.</span><span class="nx">s3_source</span><span class="err">.</span><span class="nx">arn</span>
      <span class="p">},</span>
      <span class="p">{</span>
        <span class="nx">Action</span> <span class="p">=</span> <span class="p">[</span>
          <span class="s2">"s3:AbortMultipartUpload"</span><span class="p">,</span>
          <span class="s2">"s3:DeleteObject"</span><span class="p">,</span>
          <span class="s2">"s3:GetObject"</span><span class="p">,</span>
          <span class="s2">"s3:ListBucket"</span><span class="p">,</span>
          <span class="s2">"s3:ListMultipartUploadParts"</span><span class="p">,</span>
          <span class="s2">"s3:PutObject"</span><span class="p">,</span>
          <span class="s2">"s3:GetObjectTagging"</span><span class="p">,</span>
          <span class="s2">"s3:PutObjectTagging"</span>
        <span class="p">]</span>
        <span class="nx">Effect</span>   <span class="p">=</span> <span class="s2">"Allow"</span>
        <span class="nx">Resource</span> <span class="p">=</span> <span class="s2">"${aws_s3_bucket.s3_source.arn}/*"</span>
      <span class="p">},</span>
      <span class="c1"># Permissions for the destination bucket</span>
      <span class="p">{</span>
        <span class="nx">Action</span> <span class="p">=</span> <span class="p">[</span>
          <span class="s2">"s3:GetBucketLocation"</span><span class="p">,</span>
          <span class="s2">"s3:ListBucket"</span><span class="p">,</span>
          <span class="s2">"s3:ListBucketMultipartUploads"</span>
        <span class="p">]</span>
        <span class="nx">Effect</span>   <span class="p">=</span> <span class="s2">"Allow"</span>
        <span class="nx">Resource</span> <span class="p">=</span> <span class="nx">aws_s3_bucket</span><span class="err">.</span><span class="nx">s3_destination</span><span class="err">.</span><span class="nx">arn</span>
      <span class="p">},</span>
      <span class="p">{</span>
        <span class="nx">Action</span> <span class="p">=</span> <span class="p">[</span>
          <span class="s2">"s3:AbortMultipartUpload"</span><span class="p">,</span>
          <span class="s2">"s3:DeleteObject"</span><span class="p">,</span>
          <span class="s2">"s3:GetObject"</span><span class="p">,</span>
          <span class="s2">"s3:ListBucket"</span><span class="p">,</span>
          <span class="s2">"s3:ListMultipartUploadParts"</span><span class="p">,</span>
          <span class="s2">"s3:PutObject"</span><span class="p">,</span>
          <span class="s2">"s3:GetObjectTagging"</span><span class="p">,</span>
          <span class="s2">"s3:PutObjectTagging"</span>
        <span class="p">]</span>
        <span class="nx">Effect</span>   <span class="p">=</span> <span class="s2">"Allow"</span>
        <span class="nx">Resource</span> <span class="p">=</span> <span class="s2">"${aws_s3_bucket.s3_destination.arn}/*"</span>
      <span class="p">}</span>
    <span class="p">]</span>
  <span class="p">}</span><span class="err">)</span>
<span class="p">}</span>


<span class="c1"># update the source s3 bucket policy</span>
<span class="nx">resource</span> <span class="s2">"aws_s3_bucket_policy"</span> <span class="s2">"source_bucket_policy"</span> <span class="p">{</span>
  <span class="nx">bucket</span> <span class="p">=</span> <span class="nx">aws_s3_bucket</span><span class="err">.</span><span class="nx">s3_source</span><span class="err">.</span><span class="nx">id</span>

  <span class="nx">policy</span> <span class="p">=</span> <span class="nx">jsonencode</span><span class="err">(</span><span class="p">{</span>
    <span class="nx">Version</span> <span class="p">=</span> <span class="s2">"2008-10-17"</span>
    <span class="nx">Statement</span> <span class="p">=</span> <span class="p">[</span>
      <span class="p">{</span>
        <span class="nx">Sid</span>    <span class="p">=</span> <span class="s2">"DataSyncCreateS3LocationAndTaskAccess"</span>
        <span class="nx">Effect</span> <span class="p">=</span> <span class="s2">"Allow"</span>
        <span class="nx">Principal</span> <span class="p">=</span> <span class="p">{</span>
          <span class="s2">"AWS"</span> <span class="err">:</span> <span class="s2">"${aws_iam_role.datasync_role.arn}"</span>
        <span class="p">}</span>
        <span class="nx">Action</span> <span class="p">=</span> <span class="p">[</span>
          <span class="s2">"s3:GetBucketLocation"</span><span class="p">,</span>
          <span class="s2">"s3:ListBucket"</span><span class="p">,</span>
          <span class="s2">"s3:ListBucketMultipartUploads"</span><span class="p">,</span>
          <span class="s2">"s3:AbortMultipartUpload"</span><span class="p">,</span>
          <span class="s2">"s3:DeleteObject"</span><span class="p">,</span>
          <span class="s2">"s3:GetObject"</span><span class="p">,</span>
          <span class="s2">"s3:ListMultipartUploadParts"</span><span class="p">,</span>
          <span class="s2">"s3:PutObject"</span><span class="p">,</span>
          <span class="s2">"s3:GetObjectTagging"</span><span class="p">,</span>
          <span class="s2">"s3:PutObjectTagging"</span>
        <span class="p">]</span>
        <span class="nx">Resource</span> <span class="p">=</span> <span class="p">[</span>
          <span class="nx">aws_s3_bucket</span><span class="err">.</span><span class="nx">s3_source</span><span class="err">.</span><span class="nx">arn</span><span class="p">,</span>
          <span class="s2">"${aws_s3_bucket.s3_source.arn}/*"</span>
        <span class="p">]</span>
      <span class="p">}</span>
    <span class="p">]</span>
  <span class="p">}</span><span class="err">)</span>
<span class="p">}</span>

<span class="c1"># update the destination s3 bucket policy</span>
<span class="nx">resource</span> <span class="s2">"aws_s3_bucket_policy"</span> <span class="s2">"destination_bucket_policy"</span> <span class="p">{</span>
  <span class="nx">bucket</span> <span class="p">=</span> <span class="nx">aws_s3_bucket</span><span class="err">.</span><span class="nx">s3_destination</span><span class="err">.</span><span class="nx">id</span>

  <span class="nx">policy</span> <span class="p">=</span> <span class="nx">jsonencode</span><span class="err">(</span><span class="p">{</span>
    <span class="nx">Version</span> <span class="p">=</span> <span class="s2">"2008-10-17"</span>
    <span class="nx">Statement</span> <span class="p">=</span> <span class="p">[</span>
      <span class="p">{</span>
        <span class="nx">Sid</span>    <span class="p">=</span> <span class="s2">"DataSyncCreateS3LocationAndTaskAccess"</span>
        <span class="nx">Effect</span> <span class="p">=</span> <span class="s2">"Allow"</span>
        <span class="nx">Principal</span> <span class="p">=</span> <span class="p">{</span>
          <span class="s2">"AWS"</span> <span class="err">:</span> <span class="s2">"${aws_iam_role.datasync_role.arn}"</span>
        <span class="p">}</span>
        <span class="nx">Action</span> <span class="p">=</span> <span class="p">[</span>
          <span class="s2">"s3:GetBucketLocation"</span><span class="p">,</span>
          <span class="s2">"s3:ListBucket"</span><span class="p">,</span>
          <span class="s2">"s3:ListBucketMultipartUploads"</span><span class="p">,</span>
          <span class="s2">"s3:AbortMultipartUpload"</span><span class="p">,</span>
          <span class="s2">"s3:DeleteObject"</span><span class="p">,</span>
          <span class="s2">"s3:GetObject"</span><span class="p">,</span>
          <span class="s2">"s3:ListMultipartUploadParts"</span><span class="p">,</span>
          <span class="s2">"s3:PutObject"</span><span class="p">,</span>
          <span class="s2">"s3:GetObjectTagging"</span><span class="p">,</span>
          <span class="s2">"s3:PutObjectTagging"</span>
        <span class="p">]</span>
        <span class="nx">Resource</span> <span class="p">=</span> <span class="p">[</span>
          <span class="nx">aws_s3_bucket</span><span class="err">.</span><span class="nx">s3_destination</span><span class="err">.</span><span class="nx">arn</span><span class="p">,</span>
          <span class="s2">"${aws_s3_bucket.s3_destination.arn}/*"</span>
        <span class="p">]</span>
      <span class="p">}</span>
    <span class="p">]</span>
  <span class="p">}</span><span class="err">)</span>
<span class="p">}</span>


</code></pre></div></div>

<h3 id="datasync-task-and-location">Datasync Task and Location</h3>

<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1"># create datasync source and destination location</span>
<span class="c1"># Create DataSync source location</span>
<span class="nx">resource</span> <span class="s2">"aws_datasync_location_s3"</span> <span class="s2">"source"</span> <span class="p">{</span>
  <span class="nx">s3_bucket_arn</span> <span class="p">=</span> <span class="nx">aws_s3_bucket</span><span class="err">.</span><span class="nx">s3_source</span><span class="err">.</span><span class="nx">arn</span>
  <span class="nx">subdirectory</span>  <span class="p">=</span> <span class="s2">"/"</span>
  <span class="nx">s3_config</span> <span class="p">{</span>
    <span class="nx">bucket_access_role_arn</span> <span class="p">=</span> <span class="nx">aws_iam_role</span><span class="err">.</span><span class="nx">datasync_role</span><span class="err">.</span><span class="nx">arn</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="nx">resource</span> <span class="s2">"aws_datasync_location_s3"</span> <span class="s2">"destination"</span> <span class="p">{</span>
  <span class="nx">s3_bucket_arn</span> <span class="p">=</span> <span class="nx">aws_s3_bucket</span><span class="err">.</span><span class="nx">s3_destination</span><span class="err">.</span><span class="nx">arn</span>
  <span class="nx">subdirectory</span>  <span class="p">=</span> <span class="s2">"/"</span>
  <span class="nx">s3_config</span> <span class="p">{</span>
    <span class="nx">bucket_access_role_arn</span> <span class="p">=</span> <span class="nx">aws_iam_role</span><span class="err">.</span><span class="nx">datasync_role</span><span class="err">.</span><span class="nx">arn</span>
  <span class="p">}</span>
<span class="p">}</span>

<span class="c1"># create datasync task</span>
<span class="nx">resource</span> <span class="s2">"aws_datasync_task"</span> <span class="s2">"s3_same_region"</span> <span class="p">{</span>
  <span class="nx">name</span>                     <span class="p">=</span> <span class="s2">"s3_same_region"</span>
  <span class="nx">destination_location_arn</span> <span class="p">=</span> <span class="nx">aws_datasync_location_s3</span><span class="err">.</span><span class="nx">destination</span><span class="err">.</span><span class="nx">arn</span>
  <span class="nx">source_location_arn</span>      <span class="p">=</span> <span class="nx">aws_datasync_location_s3</span><span class="err">.</span><span class="nx">source</span><span class="err">.</span><span class="nx">arn</span>

<span class="p">}</span>

</code></pre></div></div>

<h2 id="running-the-data-transfer">Running the Data Transfer</h2>

<p>After applying the Terraform configuration, the DataSync task can be triggered manually from the AWS Console or via AWS CLI. For this demo, i will upload the CSV file to the S3 source bucket using CLI. You can refer to <a href="https://docs.aws.amazon.com/cli/latest/userguide/getting-started-quickstart.html">this documentation</a> about how to set up AWS CLI.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c"># Upload a sample CSV file to the source S3 bucket</span>
aws s3 <span class="nb">cp </span>currency.csv s3://nugrohosource-testing-bucket/ <span class="nt">--profile</span> default

</code></pre></div></div>

<p>Once the file is uploaded, start the DataSync task either from the AWS Console or by using the AWS CLI. And after task execution sucessfully, you’ll see the CSV file is in the destination bucket.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Verify the file in the destination S3 bucket</span>
aws s3 <span class="nb">ls </span>s3://nugrohosource-testing-bucket/ <span class="nt">--profile</span> default


</code></pre></div></div>

<h2 id="monitoring-and-logging">Monitoring and Logging</h2>

<p>AWS DataSync integrates with Amazon CloudWatch to provide:</p>
<ul>
  <li>Task execution status</li>
  <li>Bytes transferred</li>
  <li>Error logs</li>
</ul>

<h2 id="common-errors-and-troubleshooting">Common Errors and Troubleshooting</h2>

<h3 id="access-denied-ensure-bucket-access-role-has-s3listbucket-permission">Access denied. Ensure bucket access role has s3:ListBucket permission.</h3>
<p><strong>Errors:</strong></p>
<ul>
  <li>Access denied. Ensure bucket access role has s3:ListBucket permission.</li>
</ul>

<p><strong>Solution:</strong></p>
<ul>
  <li>Ensure the IAM Role has “Action”: “sts:AssumeRole”</li>
</ul>

<p><strong>References:</strong></p>
<ul>
  <li>https://repost.aws/questions/QUPD3ZX7p3T3OQkk19QN-iEw/datasync-between-s3-buckets-failing-ensure-bucket-access-role-has-s3-listbucket-permission</li>
</ul>

<h3 id="cross-account-s3-to-s3">Cross-Account S3 to S3</h3>
<p>If you want to transfer data between s3 in the different account, you can refer to this documentation:</p>

<p><strong>References:</strong></p>
<ul>
  <li>https://docs.aws.amazon.com/datasync/latest/userguide/tutorial_s3-s3-cross-account-transfer.html</li>
  <li>https://medium.com/@nayanarora55/cross-account-s3-migration-aws-datasync-does-it-so-you-dont-have-to-9b85a23d1464</li>
</ul>]]></content><author><name>Nugroho</name></author><category term="cloud" /><category term="aws" /><category term="datasync" /><category term="s3" /><summary type="html"><![CDATA[AWS DataSync is a managed data transfer service that makes it easy to move data between on-premises storage and AWS storage services, as well as between AWS storage services themselves. In this guide, i’ll walk through the steps to move data from one S3 bucket to another S3 bucket using AWS DataSync.]]></summary></entry></feed>