Zola 2024-04-28T00:00:00+00:00 https://desipient.com/tags/metallb/atom.xml 2024-04-28T00:00:00+00:00 2024-04-28T00:00:00+00:00 Unknown https://desipient.com/blog/multiple-ingress-nginx-instances-with-metallb-and-argocd/ <p>In the process of updating my <a rel="external" href="https://kubernetes.github.io/ingress-nginx/">ingress-nginx</a> controllers to a <a href="../my-clusters-journey/">ArgoCD application</a> on my k8s cluster, I found myself needing to have multiple instances running within a single namespace and Argo application. OK, maybe I didn’t <em>need</em> to, but my sense of organisation needed to…</p> <p>Of course, the first go at setting this up was to add an Argo application sourcing the helm chart twice, and basically copy-pasting the helm values from the FAQ <a rel="external" href="https://kubernetes.github.io/ingress-nginx/user-guide/k8s-122-migration/#i-cant-use-multiple-namespaces-what-should-i-do">specifically</a> about installing multiple instances within the same namespace for the second instance. But this approach was thwarted by the documentation being wrong. Using these values results in at least one resource with a duplicate name (the controller <code>service</code> at least, but I didn’t check for more). Had I attempted the install with helm, this should have failed as the combination of name and resource type is unique within a namespace. Argo, however, will happily accept the duplicates and ignore all but the last generated.<sup class="footnote-reference" id="fr-1-1"><a href="#fn-1">1</a></sup></p> <p>So the fix? Easy - the chart provides a <code>nameOverride</code> value which can be used to override the generated names of the resources. This includes, by the way, the <code>electionID</code>, so we can drop that specification in our values.</p> <p>The resulting application definition looks a little something like the below. Note as well the annotations for the IPs requested from MetalLB. This ensure that the assigned IPs for each don’t change when moving between namespaces.</p> <pre class="giallo" style="color: #CDD6F4; background-color: #1E1E2E;"><code data-lang="yaml"><span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;"> 1</span><span style="color: #89B4FA;">apiVersion</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> argoproj.io/v1alpha1</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;"> 2</span><span style="color: #89B4FA;">kind</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> Application</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;"> 3</span><span style="color: #89B4FA;">metadata</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;"> 4</span><span style="color: #89B4FA;"> name</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> ingress-nginx</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;"> 5</span><span style="color: #89B4FA;"> namespace</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> argo</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;"> 6</span><span style="color: #89B4FA;"> finalizers</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;"> 7</span><span style="color: #9399B2;"> -</span><span style="color: #A6E3A1;"> resources-finalizer.argocd.argoproj.io</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;"> 8</span><span style="color: #89B4FA;">spec</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;"> 9</span><span style="color: #89B4FA;"> project</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> default</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">10</span><span style="color: #89B4FA;"> sources</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">11</span><span style="color: #9399B2;"> -</span><span style="color: #89B4FA;"> repoURL</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> https://kubernetes.github.io/ingress-nginx</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">12</span><span style="color: #89B4FA;"> chart</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> ingress-nginx</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">13</span><span style="color: #89B4FA;"> targetRevision</span><span style="color: #94E2D5;">:</span><span style="color: #FAB387;"> 4.10.1</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">14</span><span style="color: #89B4FA;"> helm</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">15</span><span style="color: #89B4FA;"> valuesObject</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">16</span><span style="color: #89B4FA;"> controller</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">17</span><span style="color: #89B4FA;"> service</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">18</span><span style="color: #89B4FA;"> annotations</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">19</span><span style="color: #89B4FA;"> metallb.universe.tf/loadBalancerIPs</span><span style="color: #94E2D5;">:</span><span style="color: #FAB387;"> 192.168.0.1</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">20</span><span style="color: #9399B2;"> -</span><span style="color: #89B4FA;"> repoURL</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> https://kubernetes.github.io/ingress-nginx</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">21</span><span style="color: #89B4FA;"> chart</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> ingress-nginx</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">22</span><span style="color: #89B4FA;"> targetRevision</span><span style="color: #94E2D5;">:</span><span style="color: #FAB387;"> 4.10.1</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">23</span><span style="color: #89B4FA;"> helm</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">24</span><span style="color: #89B4FA;"> valuesObject</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">25</span><span style="color: #89B4FA;"> nameOverride</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> ingress-nginx-secondary</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">26</span><span style="color: #89B4FA;"> controller</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">27</span><span style="color: #89B4FA;"> ingressClassResource</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">28</span><span style="color: #89B4FA;"> name</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> nginx-secondary</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">29</span><span style="color: #89B4FA;"> controllerValue</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> k8s.io/ingress-nginx-secondary</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">30</span><span style="color: #89B4FA;"> enabled</span><span style="color: #94E2D5;">:</span><span style="color: #FAB387;"> true</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">31</span><span style="color: #89B4FA;"> ingressClassByName</span><span style="color: #94E2D5;">:</span><span style="color: #FAB387;"> true</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">32</span><span style="color: #89B4FA;"> ingressClass</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> nginx-secondary</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">33</span><span style="color: #89B4FA;"> service</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">34</span><span style="color: #89B4FA;"> annotations</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">35</span><span style="color: #89B4FA;"> metallb.universe.tf/loadBalancerIPs</span><span style="color: #94E2D5;">:</span><span style="color: #FAB387;"> 192.168.0.2</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">36</span><span style="color: #89B4FA;"> destination</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">37</span><span style="color: #89B4FA;"> server</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> https://kubernetes.default.svc</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">38</span><span style="color: #89B4FA;"> namespace</span><span style="color: #94E2D5;">:</span><span style="color: #A6E3A1;"> ingress-nginx</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">39</span><span style="color: #89B4FA;"> syncPolicy</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">40</span><span style="color: #89B4FA;"> syncOptions</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">41</span><span style="color: #9399B2;"> -</span><span style="color: #A6E3A1;"> CreateNamespace=true</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">42</span><span style="color: #9399B2;"> -</span><span style="color: #A6E3A1;"> PruneLast=true</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">43</span><span style="color: #89B4FA;"> automated</span><span style="color: #94E2D5;">:</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">44</span><span style="color: #89B4FA;"> selfHeal</span><span style="color: #94E2D5;">:</span><span style="color: #FAB387;"> true</span></span> <span class="giallo-l"><span aria-hidden="true" class="giallo-ln" style="color: #7F849C;">45</span><span style="color: #89B4FA;"> prune</span><span style="color: #94E2D5;">:</span><span style="color: #FAB387;"> true</span></span></code></pre><section class="footnotes"> <ol class="footnotes-list"> <li id="fn-1"> <p>Yes, this is expected behaviour. Yes, I am aware that this says more about the fact that I am completely abusing the multiple source feature of ArgoCD. <a href="#fr-1-1">↩</a></p> </li> </ol> </section>