Welcome to BGP - Setting Up Your Own BGP Network: A Step-by-Step Guide (IPv6)

Learn how to configure your own BGP network using open-source tools like BIRD2, PathVector, and Routinator. This guide focuses on IPv6 and provides a practical approach to BGP routing with real-world examples.

What's BGP?

BGP stands for Border Gateway Protocol and is used to discover who is anouncing a certain IP address subnet, BGP routers communicate between them by sharing routing tables.

What concepts should I be aware?

RIR - Regional Internet Registry (AFRINIC,APNIC,ARIN,LACNIC,RIPE)
LIR - Local Internet Registry - Local Providers which are the middleman between you and the RIR
IRR - Internet Routing Registry - A database containing who owns certain objects (ASNs, Subnets, etc), managed by RIR's and big ISPs (eg.: NTT)
ASN - Autonomous System Number- Identifies a network and is represented by an AUT-NUM object in the RIR
PA - Provider Aggregatable (PA) - It's a type of address allocation, from your LIR, in IPv6 you'll probabbly get one or more for free
PI - Provider Independent (PI) - Another type of allocation, directly from the RIR - Certainly not for free
RPKI - Resource Public Key Infrastructure - In short, a way to anchor an IP Address (subnet) with an ASN using a certification path
Upstream - The entity which will give you transit (let's call it internet access)
Route Server - A BGP server that usually exchanges routes but doesn't route traffic (usually present in exchanges)
Peer - An entity to which you connect
Downstream - An entity to which you are the transit provider
Exchange - A place (can be virtual or physical) where people connect
Community - Logical group of routes/prefixes

Two routers having a conversation

sequenceDiagram
Router1-->>Router2: Hey, I'm AS123456!
Router2-->>Router1: Hey there, I'm AS234567!
Router1-->>Router2:  I'm announcing 2a00:1450:4000:/48
Router1-->>Router2:  I'm announcing 2a00:1450:4001:/48
Router1-->>Router2:  (20k more announcements)

Router2-->>Router1: Okay, I'll send you the traffic to those routes
Router2-->>Router1: I'm announcing 2a00:1450:1::/48
Router1-->>Router2: Okay, I'll send you the traffic to those routes

In this conversation, the actors were:

Router 1 - An upstream provider, knows how to route all the traffic

Router 2 - A downstream, announces it's own routes but he receives all other routes from the upstream

What do I need to get started?

First things first:

Create the essential objects: https://apps.db.ripe.net/db-web-ui/webupdates/select

Person and Maintainer Pair - This will pair the maintainer widht your RIR SSO and your 'person'

 mntner:         <FIRST_NAME>-<MIDDLE_NAME>-<LAST_NAME>-MNT
 person:         <Your Full Name>
 address:        <Your Full Address, Country included>
 e-mail:         <Your Email Address>
 phone:          <Your Phone With International Preffix>

Generic Contacts Role - This will have your Abuse & NOC contacts

 role:           AS<ASN>-CONTACT
 address:        <Your Full Address, Country included>
 e-mail:         <Your NOC Email Address>
 abuse-mailbox:  <Your Abuse Email Address>
 nic-hdl:        AUTO-1                          # RIPE will auto generate it
 mnt-by:         <Filled automatically with the previous Maintainer>

Organization - This will be the owner of your 'Internet Number' objects (ASN, Subnets, etc)

 organisation:   ORG-SS1376-RIPE
 org-name:       <Your Full Name>
 country:        <Country Code>
 org-type:       OTHER                   # For personal ASNs
 address:        <Your full address>
 e-mail:         <Your email address>
 abuse-c:        <The NIC HDL of the Generic Contact role you created before>

Choose a LIR to apply for an ASN on your local RIR, give preference to a LIR that has experience doing it:
- InfernoComms - 45£ (one-time) + 30£ (per year) for a /48 PA IPv6 subnet
- Lagrange - 15£ (one-time) - Include a /48 PA IPv6 subnet
- iFog - 60CHF (one time) + 30CHF (per year) for a /48 PA IPv6 subnet
Wait a few days, it should take about a week
Secure your resources
- Create a ROUTE/ROUTE6 object for your newly assigned subnet
```
route6:         <your_subnet>::/<subnet_size>
origin:         AS<your_asn>
mnt-by:         <your maintainer id>
```
- If you got a PA subnet from your LIR, ask him to secure it with RPKI
- If you got a PI subnet from your LIR you can certificate a path using:
- Rour RIR platform: https://my.ripe.net/#/rpki
- Your own RPKI server
Create a few more objects on your RIR (link for samples)
- AS-SET [YOUR_ASN_NUMBER]:AS-UPSTREAMS :: https://rest.db.ripe.net/RIPE/as-set/AS215153:AS-UPSTREAMS.txt
- AS-SET [YOUR_ASN_NUMBER]:AS-EXCHANGES :: https://rest.db.ripe.net/RIPE/as-set/AS215153:AS-EXCHANGES.txt
- AS-SET [YOUR_ASN_NUMBER]:AS-PEERS :: https://rest.db.ripe.net/RIPE/as-set/AS215153:AS-PEERS.txt
- AS-SET [YOUR_ASN_NUMBER]:AS-CONE :: https://rest.db.ripe.net/RIPE/as-set/AS215153:AS-CONE.txt
- AS-SET [YOUR_ASN_NUMBER]:AS-DOWNSTREAMS :: https://rest.db.ripe.net/RIPE/as-set/AS215153:AS-DOWNSTREAMS.txt

Update your new AUT-NUM to import/export the previous objects according to their purpose:

aut-num:        <dont_change_it>
as-name:        <dont_change_it>
org:            <dont_change_it>
sponsoring-org: <dont_change_it>
remarks:        ---------------------------------------------------------------
remarks:        -                       UPSTREAM                              -
remarks:        ---------------------------------------------------------------
remarks:        - We accept anything from upstreams and announce us and our   -
remarks:        - downstreams.                                                -
remarks:        ---------------------------------------------------------------
mp-import:      afi ipv6.unicast from AS<your_asn>:AS-UPSTREAMS accept ANY
mp-export:      afi ipv6.unicast to AS<your_asn>:AS-UPSTREAMS announce AS<your_asn>:AS-CONE
remarks:        ---------------------------------------------------------------
remarks:        -                       EXCHANGES                              -
remarks:        ---------------------------------------------------------------
remarks:        - We accept anything from exchanges and announce us and our   -
remarks:        - downstreams.                                                -
remarks:        ---------------------------------------------------------------
mp-import:      afi ipv6.unicast from AS<your_asn>:AS-EXCHANGES accept AS<your_asn>:AS-EXCHANGES
mp-export:      afi ipv6.unicast to AS<your_asn>:AS-EXCHANGES announce AS<your_asn>:AS-CONE
remarks:        ---------------------------------------------------------------
remarks:        -                         PEERS                               -
remarks:        ---------------------------------------------------------------
remarks:        - We accept anything from peers and announce us and our       -
remarks:        - downstreams (no transit).                                   -
remarks:        ---------------------------------------------------------------
mp-import:      afi ipv6.unicast from AS<your_asn>:AS-PEERS accept AS<your_asn>:AS-PEERS
mp-export:      afi ipv6.unicast to AS<your_asn>:AS-PEERS announce AS<your_asn>:AS-CONE
remarks:        ---------------------------------------------------------------
remarks:        -                       DOWNSTREAM                            -
remarks:        ---------------------------------------------------------------
mp-import:      afi ipv6.unicast from AS<your_asn>:AS-DOWNSTREAMS accept AS<your_asn>:AS-DOWNSTREAMS
mp-export:      afi ipv6.unicast to AS<your_asn>:AS-DOWNSTREAMS announce ANY
remarks:
remarks:        ------------------------------------------------------------------
remarks:
admin-c:        <your-person-id>
tech-c:         <your-person-id>
abuse-c:        <your-contact-id>

The idea of this approach is to keep a static version of you AUTNUM by maintaining just the ASSETS, some upstreams build their own filters using legacy scripts that might get broken with this approach, please validate with your upstreams if it's okay to use this approach or if you need to keep all the AS's in your AUTNUM for their filters to work properly.

We finished the object configuration, the objects created so far will ensure that the automations on your upstreams will allow you to announce your prefixes and your downstream's. Be aware that some upstreams configure things manually and you need to inform them every single time you (or your downstreams) add new prefixes. Avoid those upstream providers, things should be automated and subnet ownership certification should be done with RKPI/IRR objects.

Now seriously, let's get started!

In this post I'll use:

BIRD2 - as iBGP (internal) and eBGP (external) router
PathVector - a declarative configuration tool that uses yaml and generate Bird config files, it automatically validates IRR records when generating the config files and can configure your Bird to also lookup for RPKI records in real time (when routes are received from upstreams/peers).
Routinator - This will fech RPKI records from multiple sources and will aggregate them locally (I wasn't able to find reliable / single-source-of-truth RPKI servers online most of the times)

Sharing my setup

I have a core router in my main location (where I don't have many local peers unfortunatly), and edge routers in London, Madrid, Barcelona and Oporto.

graph LR
A[Lisbon] --> B((Oporto))
A[Lisbon] --> C((London1))
A[Lisbon] --> D((London2))
A[Lisbon] --> E((Madrid))
A[Lisbon] --> F((Barcelona))
B --> K{Cloudity}
B -- Tunnel--> L{FreeTunnel.ch}
E --> H{VultR}
C --> I{iFog}
D --> J{Lagrange}
F --> I{iFog}

How does everything works?

Since all my network is in Lisbon, that's were I'm announcing all my prefixes using iBGP to the edge locations, each edge location will then advertise it to the eBGP peers and upstreams.

Setting things Up

Make the initial setup of your Ubuntu server

Install the required packages: Parts from https://pathvector.io/docs/installation

curl https://repo.pathvector.io/pgp.asc > /usr/share/keyrings/pathvector.asc
echo  "deb [signed-by=/usr/share/keyrings/pathvector.asc] https://repo.pathvector.io/apt/ stable main"  > /etc/apt/sources.list.d/pathvector.list
apt update && apt install -y pathvector bgpq4 bird2

With everything installed, create a new config file /etc/pathvector.yml, inline comments to help you to understand it

asn: <your_asn_numberic_value>
router-id: <unique_identifier>                                          # Unique identification of your router (use the router's public ipv4)
bgpq-args: "-S AFRINIC,APNIC,ARIN,LACNIC,RIPE"                          # Arguments to query all RIRs
irr-query-timeout: 30
irr-server: "128.241.192.40"                                            # NTT's IRR IP
rtr-server: "<routinator_ip>:3323"                                      # Your routinator ip:port
peeringdb-api-key: "<peering_db_api_key_here>"
peeringdb-query-timeout: 30
default-route: false                                                    # We won't need a default route
merge-paths: true                                                       # Import equivalent routes (when there are multiple best matches)
source6: <one_address_assigned_in_one_of_the_routers_interfaces>        # Address to add (as src addr) when routing traffic to the next hop
kernel:
learn: true                                                           # Learn from iBGP and eBGP (internal and external routes will be added automatically)
templates:
ibgp:                                                                 # iBGP template
 announce-all:                                                       # Announce all routes to our iBGPs
 allow-local-as: true                    # Allow our own AS between connections, after all is an iBGP
 asn: <your_asn>                         # Our ASN
 direct: true                            # Specifies we connect directly to the other router (no hops between)
 enforce-first-as: false                 # We are exchanging routes without adding our AS as the first hop, so it should be false
 enforce-peer-nexthop: true              # Our neighbor should tell us he's the next hop, otherwise we won't know how to get there
 filter-rpki: false                      # We won't filter anything here, its done at the edges, not internally
 next-hop-self: true                     # We'll be the next hop for announced routes
 export: true                            # Export routes to this peer (used together with announce-all)
upstream:                                  # UPSTREAM Template
 allow-local-as: false                    # Our upstream doesn't share the same AS
 announce: [ "<your_asn>:0:15", "<your_asn>:0:16" ] # Announce our CONE (Upstreams and ourselves)
 remove-all-communities: <your_asn>           # Remove all our communities, that's our stuff, our upstream doesn't have anything to do with it
 local-pref: 50                           # The preference of our upstream
 add-on-import: [ "<your_asn>:0:12" ]         # We add all routes from our upstreams to this community
routeserver:                               # ROUTESERVER Template - For exchanges with RS
 filter-transit-asns: true                # Reject known transit ASNs, just in case someone announces THE INTERNET to the Exchange Route servers
 auto-import-limits: true                 # For each ASN we'll use the subnet limits configured in PeeringDB, just for prevention
 enforce-peer-nexthop: false              # The next hop will most likely be someone's router in the same subnet, not the route server itself
 enforce-first-as: false                  # For the same reason as the previous
 announce: [ "<your_asn>:0:15","<your_asn>:0:16" ] # We announce ourselves and our downstreams
 remove-all-communities: <your_asn>           # Remove our communities, it's our stuff
 local-pref: 90                           # Set the local preference
 add-on-import: [ "<your_asn>:0:13" ]         # Add the imports to the RouteServers community
 next-hop-self: true                      # Set this router as the next-hop when exporting routes
peer:
 filter-irr: true                         # Filter the ROUTE6 objects to prevent peers from anouncing something that doesn't belong to them
                      # (RPKI is enabled by default, that's why I didn't add the parameter)
 filter-transit-asns: true                # Be sure the peer is not anouncing kwnow Transit ASNs
 auto-import-limits: true                 # Limit the imports according to the PeeringDB config, for prevention
 auto-as-set: true                        # Get AS-SETs from peering DB to establish what the peer can announce
 announce: [ "<your_asn>:0:15","<your_asn>:0:16"]
 remove-all-communities: <your_asn>          # Remove all our communities before exchanging routes
 local-pref: 100
 add-on-import: [ "<your_asn>:0:14" ]
 next-hop-self: true                      # Set this router as the next-hop when exporting routes
downstream:
 filter-irr: true
 allow-blackhole-community: true
 filter-transit-asns: true
 auto-import-limits: true
 auto-as-set: true
 next-hop-self: true
 announce-default: true                 # You can announce the default route (if you're the only upstream for this peer) or send the full table
 remove-all-communities: <your_asn>
 local-pref: 110
 add-on-import: [ "<your_asn>:0:15" ]
peers:
"iBGP1":
 template: ibgp
 neighbors:
   - <your_other_router_ip>
"upstream1":
 asn: <your_upstream_asn>
 template: upstream
 neighbors:
   - <your_upstream_route_server_ip>

Things I might have to explain

Communities

Communities are used to aggregate routes in logical groups, you can then announce, import or export them as you intend:

your_asn:0:12 - All routes imported from your upstream - You will announce these ONLY to your downstreams
your_asn:0:13 - All routes imported from route servers (people peering with you via RS)
your_asn:0:14 - All routes imported directly from your peers
your_asn:0:15 - Your downstream's routes - You'll announce them to your upstream
your_asn:0:16 - Your own routes - You'll announce them to your upstream

Specifics

If you're using VultR you'll need to add this:

...
kernel:
  learn: true
  statics:
    "2001:19f0:ffff::1/128": "fe80::fc00:4ff:fedb:b9f3%enp1s0"  # This is an example, you'll have to route the neighbors (their RS) via you default gw%interface
...
peers:
  "VultR":
    asn: 64515                          # Their internal ASN
    template: upstream                  # Use the base upstream template
    multihop: true                      # Set it for multihop due to their config
    enforce-first-as: false             # They'r announcing the routes with their external ASN as
                                        # first in the path, so we have to ignore it (we won't see
                                        # the internal ASN in the path)
    enforce-peer-nexthop: false         # For the same reason as the first-as
    filter-bogon-asns: false
    password: "<password_here>"
    neighbors:
      - 2001:19f0:ffff::1

Conclusion

This short tutorial will let you bootstart your network quickly, PathVector will build the Bird configuration and build the proper filters by using bgp4 to fetch your peer's AUTNUMS, ASSETs and ROUTE(6) objects

What should I do next?

Check your /etc/bird/* config files, learn from what's built automatically from pathvector and correlate the pathvector config with the generated files

Troubleshoot

Check if you have routes to your desired destinations birdc show route <preffix> or ip -6 route | grep <prefix>
Check if the ssl handshakes are working (this usually means large packets are broken, mtu adjustments & mss clamping WILL BE necessary if you're using tunnels)

Last-Update: 2024/05/10