aws_networking_discussion_detailed.md
AWS Networking Deep Dive: Network Namespaces, SDN, and Link-Local Addressing
Initial System Information Investigation
First, let's check our system information:
$ uname -a Linux ip-100-123-74-46 5.15.0-1048-aws #50-Ubuntu SMP Tue Nov 14 T02:08:35 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux $ cat /etc/os-release PRETTY_NAME="Ubuntu 22.04.5 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.5 LTS (Jammy Jellyfish)"
Network Configuration Investigation
Basic Network Interface Information
$ ip addr show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 2: eth0@if77: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc noqueue state UP group default link/ether ca:69:04:9d:73:13 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 100.123.74.46/32 scope global eth0 valid_lft forever preferred_lft forever
Routing Table
$ ip route show default via 100.123.74.191 dev eth0 100.123.74.191 dev eth0 scope link $ ip route show table all default via 100.123.74.191 dev eth0 100.123.74.191 dev eth0 scope link local 100.123.74.46 dev eth0 table local proto kernel scope host src 100.123.74.46 broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1 local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1
DNS and Link-Local Investigation
$ ping -c 4 169.254.20.10 PING 169.254.20.10 (169.254.20.10) 56(84) bytes of data. 64 bytes from 169.254.20.10: icmp_seq=1 ttl=63 time=0.062 ms 64 bytes from 169.254.20.10: icmp_seq=2 ttl=63 time=0.129 ms 64 bytes from 169.254.20.10: icmp_seq=3 ttl=63 time=0.083 ms 64 bytes from 169.254.20.10: icmp_seq=4 ttl=63 time=0.058 ms $ traceroute 169.254.20.10 traceroute to 169.254.20.10 (169.254.20.10), 30 hops max, 60 byte packets 1 * * * 2 169.254.20.10 (169.254.20.10) 0.069 ms 0.099 ms 0.089 ms
Network Architecture Diagram
Physical vs. Virtual Network Layout
Physical Network Layout:
+------------------+
| AWS Infrastructure|
| (Hardware) |
+--------+---------+
|
| (Physical NIC)
|
+--------+---------+
| Hypervisor (Nitro)|
+--------+---------+
|
| (Virtual NIC)
|
+--------+---------+
| Our VM/Container |
+------------------+
Logical Network Layout
[Our Namespace] [AWS Infrastructure Namespace]
+------------------+ +-------------------------+
| | veth | |
| eth0@if77 |<----------->| if77 |
| 100.123.74.46/32 | pair | (AWS Virtual Router) |
+------------------+ +-------------------------+
|
| (SDN)
v
+------------------+
| DNS Service |
| 169.254.20.10 |
+------------------+
Network Stack Details
Layer 3 (Network)
+----------------------------------------+
| Our IP: 100.123.74.46/32 |
| Gateway: 100.123.74.191 |
| DNS: 169.254.20.10 |
+----------------------------------------+
↕
Layer 2 (Data Link)
+----------------------------------------+
| Interface: eth0@if77 |
| MAC: ca:69:04:9d:73:13 |
| MTU: 9001 |
+----------------------------------------+
↕
AWS SDN (Software Defined Networking)
+----------------------------------------+
| - Custom routing |
| - ARP handling |
| - Virtual networking |
| - Network isolation |
+----------------------------------------+
Network Namespace Investigation
Network Namespace Details
$ ip netns list $ ls -l /proc/self/ns/ total 0 lrwxrwxrwx 1 root root 0 Jan 21 12:34 cgroup -> 'cgroup:[4026531835]' lrwxrwxrwx 1 root root 0 Jan 21 12:34 ipc -> 'ipc:[4026531839]' lrwxrwxrwx 1 root root 0 Jan 21 12:34 mnt -> 'mnt:[4026531841]' lrwxrwxrwx 1 root root 0 Jan 21 12:34 net -> 'net:[4026533043]' lrwxrwxrwx 1 root root 0 Jan 21 12:34 pid -> 'pid:[4026531836]' lrwxrwxrwx 1 root root 0 Jan 21 12:34 user -> 'user:[4026531837]' lrwxrwxrwx 1 root root 0 Jan 21 12:34 uts -> 'uts:[4026531838]'
Layer 2 Configuration Details
$ ip -d link show eth0 2: eth0@if77: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc noqueue state UP mode DEFAULT group default link/ether ca:69:04:9d:73:13 brd ff:ff:ff:ff:ff:ff link-netnsid 0 veth $ ip neigh show 100.123.74.191 dev eth0 lladdr 06:74:b3:c5:7f:95 REACHABLE
Key Technical Insights
-
Network Namespace Isolation
- We're running in an isolated network namespace (net:[4026533043])
- Complete network stack isolation from host
- Custom routing rules and network behavior
-
Virtual Ethernet (veth) Setup
- eth0@if77 indicates a veth pair
- One end in our namespace
- Other end (if77) in AWS's infrastructure namespace
-
/32 Network Implementation
- Host-only IP address (100.123.74.46/32)
- No traditional broadcast domain
- Custom AWS networking allows gateway access
-
Link-Local Addressing (169.254.x.x)
- Works despite no direct Layer 2 connectivity
- Handled by AWS's SDN infrastructure
- Custom routing and ARP handling
-
AWS SDN Implementation
- Custom network virtualization
- ARP and routing handled by infrastructure
- Traditional networking rules bypassed
- High MTU (9001) for optimized performance
Special AWS Networking Features
-
Gateway Communication
- Direct communication with 100.123.74.191
- AWS infrastructure handles ARP responses
- No traditional subnet required
-
DNS Service Access
- Access to 169.254.20.10 without local network
- SDN handles routing and translation
- Low latency (< 1ms) responses
-
Network Isolation
- Complete namespace isolation
- Custom routing tables
- Controlled access to AWS services
Conclusion
This investigation reveals how AWS implements a sophisticated network virtualization layer that:
- Bypasses traditional networking constraints
- Provides secure and isolated networking
- Enables seemingly impossible network configurations
- Maintains high performance and reliability
The combination of network namespaces, veth pairs, and SDN creates a flexible and secure networking environment that can support advanced AWS features while maintaining isolation and performance.
Leave a Comment