Feature: Heqet v2

release_v2
nold 2021-09-09 20:24:50 +02:00 committed by Nold
parent 412f35972e
commit 853717a8de
45 changed files with 697 additions and 2536 deletions

3
.helmignore Normal file
View File

@ -0,0 +1,3 @@
*.swp
old/
.git

View File

@ -1,5 +0,0 @@
apiVersion: v2
appVersion: "0.3"
description: Heqet-Bootstrap ArgoCD App-of-Apps Boilerplate
name: heqet
version: 0.3.0

View File

@ -1,9 +0,0 @@
# Build Docs
IMAGE=squidfunk/mkdocs-material
default: build
build:
docker run --rm -it -v ${PWD}:/docs ${IMAGE} build
dev:
docker run --rm -it -p 8000:8000 -v ${PWD}:/docs ${IMAGE}

View File

@ -2,15 +2,16 @@
*Heqet (Egyptian ḥqt, also ḥqtyt "Heqtit") is an Egyptian goddess of fertility.*
Heqet is my attempt to make Kubernetes GitOps Deployments as easy as possible. It reduces the need of configuration by generating the required Application definitions for you. Heqet heavily relies on a Helm-Chart which will generate all applications, namespaces & more using ArgoCDs [App-of-Apps-Pattern](https://argoproj.github.io/argo-cd/operator-manual/cluster-bootstrapping/).
Heqet is my attempt to make Kubernetes GitOps Deployments as easy as possible. It reduces the need of configuration by generating the required Kubernetes resource definitions for you. Heqet heavily relies on a Helm-Chart which will generate all ArgoCD-Applications, -Projects, Namespaces & more using Argo-CDs [App-of-Apps-Pattern](https://argoproj.github.io/argo-cd/operator-manual/cluster-bootstrapping/).
## Keyfeatures
* Easy Setup [Sane Kubernetes cluster + PVC-StorageClass]
* Easy application definition & configuration
* Easy Setup [Just requires a sane Kubernetes cluster + ArgoCD + PVC-StorageClass]
* Easy / DRY application definition & configuration
* Follows the GitOps principles
* Deploy a whole application environment or cluster from a singe git-repo
* Addons like generation of `VaultSecret` and `NetworkPolicy` resources
**This project is still in a very early stage of development, but feel free to try it out & contribute!**
**This project is still in a very early stage of development, but feel free to try it out, give feedback, create an issue and contribute!**
## Overview
@ -18,22 +19,35 @@ Heqet is my attempt to make Kubernetes GitOps Deployments as easy as possible. I
## Components & Configuration
Core component is `ArgoCD` which will deploy Heqet & also your apps! All you need is a git-repo & k8s cluster.
Core component is `ArgoCD` which will deploy Heqet & also your other apps! All you need is a git-repo & k8s cluster.
The heqet Helm-Chart will generate ArgoCD Applications, namespaces and if required vault Secrets. All you need to do if add your Helm-Application to heqet's `values.yaml`.
The heqet Helm-Chart will generate ArgoCD-Applications & -Projects, Namespaces and if required `VaultSecret`s, `NetworkPolicies`, Argo-CD Repositories and more.
The configuration is seperated in different files & directories:
* `projects/` - This directory contains all your Application/Project config
* `name-of-project/` - This directory name represents the name of our project
* `project.yaml` - The most important config, containing all our applications of this project
* `values/` - Every app in our project can have it's own `values.yaml` here, named: `name-of-app.yaml`
* `name-of-app.yaml` - Values file for the application "name-of-app"
* `resources/` - This directory contains all global config, like NetworkPolcies, Repos
* `manifests/` - Can be used for static YAML-Manifests
If more configuration values are required, simply throw your applications `values.yaml` into heqets `values.d` folder, named as the application [e.g. `values.d/argocd.yaml`.
## Installation
Installing heqet can't be simpler, after configuring your apps, argocd and pushing it to your git repo:
Installing heqet can't be simpler:
0. Install Argo-CD on your cluster & set it up to your needs
1. Configure `manifests/heqet-apps.yaml` to match your Setup
2. `kubectl apply -f manifests/argocd.yaml`
3. `kubectl apply -f manifests/heqet-apps.yaml`
ArgoCD will start and bootstrap heqet.
2. `kubectl apply -f manifests/heqet-apps.yaml`
3. Create your configuration in `projects` and `resources` folders
## Example Configuration
Check out the `test`-branch of the heqet repository for the latest deployment configuration in my test environment.
Check out the `hive`-branch of this repository for the latest deployment configuration in my homelab environment.
## Docs
Check out the full documentation: [here](https://nold360.github.io/heqet)

View File

@ -1,20 +0,0 @@
#!/bin/bash
if kubectl get nodes | grep -q '^gke-' ; then
echo "[GKE] Ensure we are Cluster-Admin..."
kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole=cluster-admin \
--user=$(gcloud config get-value core/account) || exit 1
fi
echo "Installing ArgoCD..."
helm repo add argo https://argoproj.github.io/argo-helm
kubectl create ns argocd
helm install argo argo/argo-cd --namespace argocd
echo
echo "Bootstrapping Heqet Apps..."
kubectl apply -n argocd -f manifests/heqet-apps.yaml
exit 0

View File

@ -1,7 +0,0 @@
#!/bin/bash
echo "ArgoCD 'admin': $(kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2)"
echo
echo "Vault:"
kubectl logs vault-0 -n vault | egrep 'Unseal Key|Root Token'
exit 0

View File

@ -1,29 +0,0 @@
#!/bin/bash
# Generate Vault Service-Account for Apps & preseed data
function v {
echo "vault $@"
kubectl exec -it vault-0 -n vault -- vault $@
}
v auth enable kubernetes
v write auth/kubernetes/config \
token_reviewer_jwt="\$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
kubernetes_host="https://\${KUBERNETES_PORT_443_TCP_ADDR}:443" \
kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
v secrets enable -path=heqet kv-v2
v policy write "heqet-app" - <<EOF
path "heqet/*/*" {
capabilities = ["read"]
}
EOF
v write auth/kubernetes/role/heqet-app \
bound_service_account_names=vault-secrets-operator \
bound_service_account_namespaces=vault-secrets-operator \
policies=heqet-app \
ttl=6h
# Passwort: argocd
v kv put heqet/argocd/argocd-secret admin.password='$2y$12$FP8OlsVj5pOOqRAhI4XPoev1STaW01uUEZGcNPQtVZmpacebNhj9i' server.secretkey="pDYAWK2mHa68GwwVPAsQOsG/SUT8iIo3S3FXYUWf2qM="

1
charts/heqet/.helmignore Normal file
View File

@ -0,0 +1 @@
*.swp

5
charts/heqet/Chart.yaml Normal file
View File

@ -0,0 +1,5 @@
apiVersion: v2
appVersion: "1.0"
description: Simple ArgoCD App-of-Apps Bootstrapping Boilerplate
name: heqet
version: 2.0.0

View File

@ -0,0 +1,77 @@
# NetworkPolicies predefinition
# rules can be added to groups. Groups or rules can be applied to projects.
#
networkPolicy:
config:
# Generate NetworkPolicy to allow communication inside of the project namespace?
# Only gets applied when other networkpolices are active on the project
allowNamespace: true
# Default rules/groups to apply to projects
default:
groups: []
rules: []
# Group of rules
groups:
# Name: "internet", deny all but dns, proxy and ingress
internet:
- allow-dns
- allow-proxy
- allow-ingress
rules:
# Allow DNS to kube-system Namespaces, deny everything else
allow-dns:
podSelector: {}
policyTypes:
- Egress
egress:
- ports:
- port: 53
protocol: UDP
to:
- namespaceSelector:
matchLabels:
name: kube-system
# Allow access to Kube-API
allow-kubeapi:
podSelector: {}
policyTypes:
- Egress
egress:
- ports:
- port: 443
protocol: TCP
to:
- namespaceSelector:
matchLabels:
name: kube-system
# Allow access to internet proxy
allow-proxy:
podSelector: {}
policyTypes:
- Egress
egress:
- ports:
- port: 80
protocol: TCP
- port: 3128
protocol: TCP
to:
- namespaceSelector:
matchLabels:
app.heqet.gnu.one/project: proxy
# Allow access from ingress-external
allow-ingress:
podSelector: {}
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
app.heqet.gnu.one/project: ingress

View File

@ -0,0 +1,16 @@
# Dict of helm or git repos we want to add to ArgoCD
# Parameters:
# name-of-repo:
# url: https://...
# type: [default: helm | git]
#
repos:
argo:
url: https://argoproj.github.io/argo-helm
bitnami:
url: https://charts.bitnami.com/bitnami
k8s-at-home:
url: https://k8s-at-home.com/charts
heqet:
url: https://github.com/nold360/heqet
type: git

View File

@ -0,0 +1,78 @@
{{/* Heqet Main Functions */}}
{{/*
Collect resource configs & saves them in $.resources
*/}}
{{- define "heqet.resources" }}
{{- $resources := dict }}
{{- range $path, $_ := $.Files.Glob "resources/*.y*ml" }}
{{- $res := $.Files.Get $path | fromYaml | default dict }}
{{- $_ := deepCopy $res | merge $resources }}
{{- end -}}
{{- $_ := set $ "resources" $resources }}
{{- end -}}
{{/*
Collect project, app & resource configs. After that template yaml files
*/}}
{{- define "heqet.apps" }}
{{- range $path, $_ := .Files.Glob "projects/*/*.y*ml" }}
{{- $project := $.Files.Get $path | fromYaml | default dict }}
{{- $_ := set $project.config "name" ($project.config.name | default (base (dir $path))) -}}
{{/* Generate ArgoCD project */}}
{{- include "heqet.template.project" $project.config -}}
{{/* Generate single project namespace */}}
{{- if not (hasKey $project.config "namespace") -}}
{{- $_ := set $project.config "namespace" $project.config.name }}
{{- end -}}
{{- include "heqet.template.namespace" $project.config -}}
{{/* Collect project NetworkPolicies */}}
{{- if hasKey $project.config "networkPolicy" }}
{{- $data := dict "project" $project "networkPolicy" $.resources.networkPolicy }}
{{- include "heqet.addon.networkPolicy" $data -}}
{{- include "heqet.template.networkPolicy" $project -}}
{{- end -}}
{{/* Generate App Configuration */}}
{{- $currentScope := . -}}
{{- range $app := $project.apps -}}
{{/* Merge project & defaults config into app config */}}
{{- $_ := deepCopy $project.config | merge $app }}
{{- $_ := deepCopy $.Values.defaults | merge $app }}
{{- $_ := set $app "project" ($project.config.name | default (base (dir $path))) }}
{{- $_ := set $app "isApplication" true -}}
{{/* Resolve repoURL from repo name, if repoURL is not set */}}
{{- if and (hasKey $app "repo") (not (hasKey $app "repoURL")) }}
{{- if hasKey $.resources.repos $app.repo }}
{{- $repo := (get $.resources.repos $app.repo) }}
{{- $_ := set $app "repoURL" $repo.url }}
{{- else }}
{{- fail (printf "Repository with name '%s' could not be found in resource config." $app.repo) }}
{{- end }}
{{- end -}}
{{/* Generate Namespace for app, if requested */}}
{{- if and (not (hasKey $app "existingNamespace")) (ne $app.namespace $project.config.namespace) }}
{{- $_ := set $app "namespace" ($app.namespace | default $app.name) }}
{{- include "heqet.template.namespace" $app }}
{{- else if hasKey $app "existingNamespace" }}
{{- $_ := set $app "namespace" ($app.existingNamespace) }}
{{- end -}}
{{/* Collect value file & add values to app dict */}}
{{- range $value_file, $_ := $.Files.Glob (printf "%s/values/%s.y*ml" (dir $path) $app.name ) }}
{{- with $currentScope }}
{{- $values := $.Files.Get $value_file | fromYaml | default dict }}
{{- $_ := set $app "values" $values -}}
{{- end }}
{{- end -}}
{{- include "heqet.template.app" $app }}
{{- end }}
{{- end }}
{{- end }}

View File

@ -0,0 +1,37 @@
{{/*
Resolve project NetworkPolicies from rules and groups; then append config.networkPolicy.policies to project-hash
Params:
dict:
project: # Heqet project hash
networkPolicy # NetworkPolicy Ressource Config
*/}}
{{- define "heqet.addon.networkPolicy" }}
{{- if hasKey .project.config "networkPolicy" }}
{{- if not (hasKey .project.config.networkPolicy "rules") }}
{{- $_ := set .project.config.networkPolicy "rules" list }}
{{- end }}
{{- if not (hasKey .project.config.networkPolicy "config") }}
{{- $_ := set .project.config.networkPolicy "config" .networkPolicy.config }}
{{- end }}
{{- $rules := .project.config.networkPolicy.rules }}
{{- range $group := .project.config.networkPolicy.groups }}
{{- if hasKey $.networkPolicy.groups $group }}
{{- range $rule := (get $.networkPolicy.groups $group) }}
{{- $rules = append $rules $rule }}
{{- end }}
{{- $_ := set $.project.config.networkPolicy "rules" $rules }}
{{- end }}
{{- end }}
{{/* Translate project networkPolicy.rules into NetworkPolicy */}}
{{- $policies := dict }}
{{- range .project.config.networkPolicy.rules }}
{{- if hasKey $.networkPolicy.rules . }}
{{- $_ := set $policies . (get $.networkPolicy.rules .) }}
{{- end -}}
{{- end -}}
{{- $_ := set .project.config.networkPolicy "policies" $policies -}}
{{- end }}
{{- end }}

View File

@ -0,0 +1,6 @@
{{/*
This is the main template, which includes the impotant templates
*/}}
{{- include "heqet.resources" $ -}}
{{- include "heqet.apps" $ -}}
{{- include "heqet.template.repository" $ }}

View File

@ -0,0 +1,5 @@
{{/* Collect local manifests */}}
{{- range $path, $_ := $.Files.Glob "resources/manifests/*.y*ml" }}
---
{{ $.Files.Get $path }}
{{- end }}

View File

@ -0,0 +1,50 @@
{{/* Main Template generating ArgoCD Apps */}}
{{- define "heqet.template.app" }}
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: {{ .name | quote }}
namespace: argocd
labels:
app.heqet.gnu.one/name: {{ .name }}
app.heqet.gnu.one/project: {{ .project }}
app.kubernetes.io/name: {{ .name }}
app.kubernetes.io/part-of: heqet
{{- with .labels }}{{ toYaml . | nindent 4}}{{ end }}
annotations:
argocd.argoproj.io/sync-wave: {{ .syncWave | default "0" | quote }}
{{- with .annotations }}{{ toYaml . | nindent 4}}{{ end }}
spec:
project: {{ .project }}
destination:
namespace: {{ .namespace }}
server: {{ .server }}
source:
path: {{ .path | default "" | quote }}
repoURL: {{ .repoURL | quote }}
targetRevision: {{ .targetRevision | quote }}
{{- with .chart }}
chart: {{ . | quote }}
{{- end }}
helm:
{{- with .parameters }}
parameters:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .values }}
values: |
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .automated }}
syncPolicy:
automated:
prune: {{ .prune | default "false" }}
selfHeal: {{ .selfHeal | default "false" }}
{{- end }}
{{- with .ignoreDiff }}
ignoreDifferences:
{{ .ignoreDiff | toYaml | nindent 4 }}
{{- end }}
{{- with .secrets }}{{- include "heqet.template.secrets" $ }}{{- end }}
{{- end }}

View File

@ -0,0 +1,18 @@
{{/* for every app, or [if set] for one project: */}}
{{- define "heqet.template.namespace" }}
---
apiVersion: v1
kind: Namespace
metadata:
name: {{ .namespace | default .name | quote }}
labels:
{{- if .isApplication }}
app.heqet.gnu.one/name: {{ .name }}
{{- end }}
app.heqet.gnu.one/project: {{ .project | default .name }}
project.heqet.gnu.one/name: {{ .name }}
{{- with .labels }}{{- toYaml . | nindent 4}}{{- end }}
annotations:
argocd.argoproj.io/sync-wave: "-42"
{{- with .annotations }}{{ toYaml . | nindent 4}}{{- end }}
{{- end -}}

View File

@ -0,0 +1,45 @@
{{/* Generate NetworkPolicy Rules */}}
{{- define "heqet.template.networkPolicy" -}}
{{- range $key, $value := $.config.networkPolicy.policies }}
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: {{ $.config.name }}-{{ $key }}
namespace: {{ $.config.namespace | default $.config.name }}
labels:
app.heqet.gnu.one/project: {{ $.config.name }}
annotations:
argocd.argoproj.io/sync-wave: "-1"
spec:
{{- $value | toYaml | nindent 2 }}
{{- end -}}
{{- if $.config.networkPolicy.config.allowNamespace }}
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-project-namespace-{{ $.config.name }}
namespace: {{ $.config.namespace | default $.config.name }}
labels:
app.heqet.gnu.one/project: {{ $.config.name }}
annotations:
argocd.argoproj.io/sync-wave: "-1"
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
app.heqet.gnu.one/project: {{ $.config.name }}
ingress:
- from:
- namespaceSelector:
matchLabels:
app.heqet.gnu.one/project: {{ $.config.name }}
{{- end }}
{{- end }}

View File

@ -0,0 +1,22 @@
{{- define "heqet.template.project" }}
---
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: {{ .name }}
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "-10"
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
description: {{ .description | default "Application Project" }}
clusterResourceWhitelist:
- group: '*'
kind: '*'
destinations:
- namespace: '*'
server: '*'
sourceRepos:
- '*'
{{- end }}

View File

@ -0,0 +1,15 @@
{{- define "heqet.template.repository" }}
{{- range $name, $config := $.resources.repos }}
---
apiVersion: v1
kind: Secret
metadata:
name: {{ $config.type | default "helm" }}-repo-{{ $name }}
namespace: argocd
labels:
argocd.argoproj.io/secret-type: repository
stringData:
url: {{ $config.url }}
type: {{ $config.type | default "helm" }}
{{- end }}
{{- end }}

View File

@ -0,0 +1,24 @@
{{/* Generate VaultSecrets */}}
{{- define "heqet.template.secrets" -}}
{{- $context := . }}
{{- with $context }}
{{- range .secrets }}
---
apiVersion: ricoberger.de/v1alpha1
kind: VaultSecret
metadata:
name: {{ .name }}
namespace: {{ $context.namespace | default $context.existingNamespace | default $context.name | quote }}
labels:
app.heqet.gnu.one/name: {{ $context.name }}
app.heqet.gnu.one/project: {{ $context.project }}
annotations:
argocd.argoproj.io/sync-wave: "-1"
spec:
keys:
{{- toYaml .keys | nindent 2}}
path: heqet/{{ .fromApp | default $context.name }}/{{ .name }}
type: {{ $context.type | default "Opaque" }}
{{ end }}
{{- end }}
{{- end }}

13
charts/heqet/values.yaml Normal file
View File

@ -0,0 +1,13 @@
# Default values that are used for creating ArgoCD `Application` definitions
# You can add all supported heqet config options here
defaults:
# ArgoCD Project
project: "default"
# Kubernetes Cluster
server: https://kubernetes.default.svc
# Automated sync settings
automated:
prune: true
selfHeal: true

View File

@ -1,220 +0,0 @@
---
# Source: vault-secrets-operator/templates/custom-resource-definition.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: vaultsecrets.ricoberger.de
labels:
app.kubernetes.io/name: vault-secrets-operator
helm.sh/chart: vault-secrets-operator-1.14.2
app.kubernetes.io/instance: vault-secrets-operator
app.kubernetes.io/managed-by: Helm
spec:
group: ricoberger.de
names:
kind: VaultSecret
listKind: VaultSecretList
plural: vaultsecrets
singular: vaultsecret
scope: Namespaced
versions:
- additionalPrinterColumns:
- description: Indicates if the secret was created/updated successfully
jsonPath: .status.conditions[?(@.type=="SecretCreated")].status
name: Succeeded
type: string
- description: Reason for the current status
jsonPath: .status.conditions[?(@.type=="SecretCreated")].reason
name: Reason
type: string
- description: Message with more information, regarding the current status
jsonPath: .status.conditions[?(@.type=="SecretCreated")].message
name: Message
type: string
- description: Time when the condition was updated the last time
jsonPath: .status.conditions[?(@.type=="SecretCreated")].lastTransitionTime
name: Last Transition
type: date
- description: Time when this VaultSecret was created
jsonPath: .metadata.creationTimestamp
name: Age
type: date
name: v1alpha1
schema:
openAPIV3Schema:
description: VaultSecret is the Schema for the vaultsecrets API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: VaultSecretSpec defines the desired state of VaultSecret
properties:
isBinary:
description: isBinary is a flag indicates if data stored in vault
is binary data. Since vault does not store binary data natively,
the binary data is stored as base64 encoded. However, same data
get encoded again when operator stored them as secret in k8s which
caused the data to get double encoded. This flag will skip the base64
encode which is needed for string data to avoid the double encode
problem.
type: boolean
keys:
description: Keys is an array of Keys, which should be included in
the Kubernetes secret. If the Keys field is ommitted all keys from
the Vault secret will be included in the Kubernetes secret.
items:
type: string
type: array
path:
description: Path is the path of the corresponding secret in Vault.
type: string
reconcileStrategy:
description: ReconcileStrategy defines the strategy for reconcilation.
The default value is "Replace", which replaces any existing data
keys in a secret with the loaded keys from Vault. The second valid
value is "Merge" wiche merges the loaded keys from Vault with the
existing keys in a secret. Duplicated keys will be replaced with
the value from Vault. Other values are not valid for this field.
type: string
secretEngine:
description: SecretEngine specifies the type of the Vault secret engine
in which the secret is stored. Currently the 'KV Secrets Engine
- Version 1' and 'KV Secrets Engine - Version 2' are supported.
The value must be 'kv'. If the value is omitted or an other values
is used the Vault Secrets Operator will try to use the KV secret
engine.
type: string
templates:
additionalProperties:
type: string
description: Templates, if not empty will be run through the the Go
templating engine, with `.Secrets` being mapped to the list of secrets
received from Vault. When omitted set, all secrets will be added
as key/val pairs under Secret.data.
type: object
type:
description: Type is the type of the Kubernetes secret, which will
be created by the Vault Secrets Operator.
type: string
vaultNamespace:
description: 'VaultNamespace can be used to specify the Vault namespace
for a secret. When this value is set, the X-Vault-Namespace header
will be set for the request. More information regarding namespaces
can be found in the Vault Enterprise documentation: https://www.vaultproject.io/docs/enterprise/namespaces'
type: string
vaultRole:
description: VaultRole can be used to specify the Vault role, which
should be used to get the secret from Vault. If the vaultRole property
is set a new client with the specified Vault Role will be created
and the shared client is ignored. If the operator is configured
using the token auth method this property has no effect.
type: string
version:
description: Version sets the version of the secret which should be
used. The version is only used if the KVv2 secret engine is used.
If the version is omitted the Operator uses the latest version of
the secret. If the version omitted and the VAULT_RECONCILIATION_TIME
environment variable is set, the Kubernetes secret will be updated
if the Vault secret changes.
type: integer
required:
- path
- type
type: object
status:
description: VaultSecretStatus defines the observed state of VaultSecret
properties:
conditions:
items:
description: "Condition contains details for one aspect of the current
state of this API Resource. --- This struct is intended for direct
use as an array at the field path .status.conditions. For example,
type FooStatus struct{ // Represents the observations of a
foo's current state. // Known .status.conditions.type are:
\"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type
\ // +patchStrategy=merge // +listType=map // +listMapKey=type
\ Conditions []metav1.Condition `json:\"conditions,omitempty\"
patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"`
\n // other fields }"
properties:
lastTransitionTime:
description: lastTransitionTime is the last time the condition
transitioned from one status to another. This should be when
the underlying condition changed. If that is not known, then
using the time when the API field changed is acceptable.
format: date-time
type: string
message:
description: message is a human readable message indicating
details about the transition. This may be an empty string.
maxLength: 32768
type: string
observedGeneration:
description: observedGeneration represents the .metadata.generation
that the condition was set based upon. For instance, if .metadata.generation
is currently 12, but the .status.conditions[x].observedGeneration
is 9, the condition is out of date with respect to the current
state of the instance.
format: int64
minimum: 0
type: integer
reason:
description: reason contains a programmatic identifier indicating
the reason for the condition's last transition. Producers
of specific condition types may define expected values and
meanings for this field, and whether the values are considered
a guaranteed API. The value should be a CamelCase string.
This field may not be empty.
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
type: string
status:
description: status of the condition, one of True, False, Unknown.
enum:
- "True"
- "False"
- Unknown
type: string
type:
description: type of condition in CamelCase or in foo.example.com/CamelCase.
--- Many .condition.type values are consistent across resources
like Available, but because arbitrary conditions can be useful
(see .node.status.conditions), the ability to deconflict is
important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
type: string
required:
- lastTransitionTime
- message
- reason
- status
- type
type: object
type: array
type: object
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

14
docs/Filestructure.md Normal file
View File

@ -0,0 +1,14 @@
# Directory & Filestructure
The configuration is seperated in different files & directories:
* `projects/` - This directory contains all your Application/Project config
* `name-of-project/` - This directory name represents the name of our project
* `project.yaml` - The most important config, containing all our applications of this project
* `values/` - Every app in our project can have it's own `values.yaml` here, named: `name-of-app.yaml`
* `name-of-app.yaml` - Values file for the application "name-of-app"
* `resources/` - This directory contains all global config, like NetworkPolcies, Repos
* `manifests/` - Can be used for static YAML-Manifests
## Overview
![Heqet Overview](https://nold360.github.io/heqet/assets/heqet-directory-overview.jpg)

12
docs/Getting-Started.md Normal file
View File

@ -0,0 +1,12 @@
# Getting Started
## Fork heqet
Since currently there is no way i know of, to seperate heqets config from it's code. I guess it's best you fork the whole github repo & start your config in `charts/heqet`. This way you should be able to "easily" rebase the fork on the current upstream in case of code changed. [If you don't mess with the code itself that is].
## Creating a new project
Copy the `example/project` folder to `charts/heqet/projects/name-of-your-project`. It contains a template for the `project.yaml` and also the `values` directory.
In the `values` directory you can simple create a new .yaml file, this the name of the app it belongs to. [same name as defined in the `project.yaml` by you.

2
docs/addons/index.md Normal file
View File

@ -0,0 +1,2 @@
# Addons
Heqet contains a "addon" feature which will create additional resources for you or further simplefy configuration by abstraction.

View File

@ -0,0 +1,131 @@
# NetworkPolicy
NetworkPolicies allow you to control/deny access to or from your application. If you want to learn how NetworkPolices work, check out the [Kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/network-policies/).
## Define Rules
Next you can create/predefine NetworkPolices in `resources/networkpolicy.yaml` like this:
```yaml
networkrPolicy:
# global config options
config:
# Generate NetworkPolicy to allow communication inside of the project namespace?
# Only gets applied when other networkpolices are active on the project
allowNamespace: true
# policies that get applied by default
defaults:
groups: []
rules: []
# NetworkPolices can be grouped:
groups: []
# Here are our policy definitions:
rules:
deny-everything:
podSelector: {}
policyTypes:
- Egress
- Ingress
```
This rule will deny all out- [Egress] and Ingoing [Ingress] network traffic.
We can define more rules ofcurse [even if 'deny-everything' is already included in 'allow-dns']:
``` yaml
networkPolicy:
# global config options
config:
# Generate NetworkPolicy to allow communication inside of the project namespace?
# Only gets applied when other networkpolices are active on the project
allowNamespace: true
# policies that get applied by default
defaults:
groups:
- insecure
rules: []
# NetworkPolices can be grouped:
groups:
insecure:
- deny-everything
- allow-dns
# Here are our policy definitions:
rules:
deny-everything:
podSelector: {}
policyTypes:
- Egress
- Ingress
allow-dns:
podSelector: {}
policyTypes:
- Egress
egress:
- ports:
- port: 53
protocol: UDP
to:
- namespaceSelector: {}
```
This will create a group of both of our rules & apply them by default to all projects.
## Apply NetworkPolicies
Finally we can apply the NetworkPolicy to our application project. Currently Heqet is not able to apply NetworkPolcies only to a single App of an project!
*from project.yaml*:
``` yaml
config:
name: secure-project
description: I like it secure!
networkPolicy:
rules:
- deny-everything
- allow-dns
groups:
- special-group
apps:
- name: my-secure-app
[...]
```
## Allow communication between Heqet projects
Here is a simple way to apply rules containing other heqet apps. Lets say we have two apps:
``` yaml
config:
name: project-one
networkPolicy:
rules:
- deny-project-two
apps:
[...]
```
So we want to deny access from `project-one` to `project-two`. In this case we can use a label that's automatically applied by heqet to every application namespace: `app.heqet.gnu.one/project`
Our policy could look something like this:
``` yaml
networkPolicy:
rules:
deny-project-two:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
app.heqet.gnu.one/project: project-two
```

40
docs/addons/repos.md Normal file
View File

@ -0,0 +1,40 @@
# Repositories
Heqet can add Helm & Git repositories to Argo-CD & resolve the repoURL in your config for you. Private repos are not supported [yet].
## Configuration
Here is a simple example of the configuration file `resources/repos.yml`:
```yaml
# Dict of helm or git repos we want to add to ArgoCD
# Parameters:
# name-of-repo:
# url: https://...
# type: [default: helm | git]
#
repos:
bitnami:
url: https://charts.bitnami.com/bitnami
## default: helm
#type: helm
heqet:
url: https://git.nold.in/nold/heqet
type: git
```
## Using Repos
Here is a snipped from a `project.yml`:
```yaml
config:
name: myproject
apps:
- name: myapp
repo: bitnami
chart: superappchart
```
As you might guess, the option `repo: bitnami` gets resolved to the `url` in our repos configuration.

Binary file not shown.

After

Width:  |  Height:  |  Size: 110 KiB

View File

@ -1,35 +1,28 @@
## Application Definition
Here is a list of available configuration options inside the `apps` array of heqets `values.yaml`.
### Required
# Application Config
## Required
| Parameter | Type | Example | Description |
|-----------|--------|---------|-------------|
| name | string | `"argocd"` | Name of your application & namespace [if not specified] |
| repoURL | string | `"https://github.com/nold360/heqet"` | URL to git or Helmchart repo |
| path | string | `"charts/heqet"` | Path to chart if using git in `repoURL` |
| or `repo` | string | `"heqet"` | Name of a predefinied Helm/Git-Repo |
| path | string | `"charts/heqet"` | Path to chart if using git as source repo |
| chart | string | `"heqet"` | Chart name [ only use either `path` or `chart` ] |
| targetRevision | string | `"1.2.3"` or `"master"` | Version of Helm-Chart or Branch/Tag of git |
### Optional
## Optional
| Parameter | Type | Default | Example | Description |
|-----------|--------|---------|---------|-------------|
| disabled | bool | false | `true` | Disable App |
| existingNamespace | string | none | `"default"` | Don't create namespace, instead use an existing one |
| namespace | string | .Values.name | `"superns"` | Name of application namespace |
| annotations | hash | | `my.anno.org/stuff: is-awesome` | Namespace annotations |
| namespace | string | Namespace of project | `"superns"` | Name of application namespace |
| annotations | hash | | `my.anno.org/stuff: is-awesome` | Kubernetes Resource annotations |
| syncWave | string | `"0"` | `"-2" | ArgoCD SyncWave |
| project | string | `"heqet"` | `"myproject"` | Name of ArgoCD Project |
| server | string | `"https://kubernetes.default.svc"` | `https://my.external.cluster:8443` | K8s Cluster to deploy to |
| prune | bool | `false` | `true` | ArgoCD automatic prune app |
| selfHeal | bool | `false` | `true` | ArgoCD automatic self-heal app |
| automated.prune | bool | `false` | `true` | ArgoCD automatic prune app |
| automated.selfHeal | bool | `false` | `true` | ArgoCD automatic self-heal app |
| ignoreDiff | array | | See ArgoCD docs | ArgoCD [ignoreDifferences](https://argoproj.github.io/argo-cd/user-guide/diffing/)
| parameters | array | |- name: ingress.host<br>value: awesome.url | Parameters override values of app |
## Custom Resource Definitions
CRDs might be required before applying application configuration. If so, copy the `crd.yaml` into the `/crds`-Directory.
## Full Example
Check out the `test`-Branch of this repo for my current testing setup.
Check out the `hive`-Branch of this repo for my current homelab setup.

17
docs/config/project.md Normal file
View File

@ -0,0 +1,17 @@
# Project Definition
Here is a list of available configuration options inside the `apps` array of heqets `values.yaml`.
## Project Config
Project configuration parameters will be merged into application parameters. So basically all application parameters can be used here.
Special project parameters:
| Parameter | Type | Default | Example | Description |
|-----------|--------|---------|---------|-------------|
| name | string | Name of project directory | my-project | Name of project in Argo-CD |
| namespace | string | Name of project | myspace | Name of default Namespace of projects apps |
| description | string | None | "My great project" | Description of project in Argo-CD |
| networkPolicy | dict | None | See [addons/networkpolicy](https://nold360.github.io/heqet/addons/networkpolicy) | |

View File

@ -1,2 +0,0 @@
# Generators
Heqet contains a "generators" feature which will create additional resources for you.

View File

@ -1,94 +0,0 @@
# NetworkPolicy
NetworkPolicies allow you to control/deny access to or from your application. If you want to learn how NetworkPolices work, check out the [Kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/network-policies/).
## Active Generator
To activate the NetPol generator we simply enable it in our `values.yaml`:
``` yaml
generators:
networkpolicy: true
```
## Define Rules
Next you can create/predefine NetworkPolices in `values.yaml` like this:
```yaml
networkpolicies:
deny-everything:
podSelector: {}
policyTypes:
- Egress
- Ingress
```
This rule will deny all out- [Egress] and Ingoing [Ingress] network traffic.
We can define more rules ofcurse [even if 'deny-everything' is already included in 'allow-dns']:
``` yaml
networkpolicies:
deny-everything:
podSelector: {}
policyTypes:
- Egress
- Ingress
allow-dns:
podSelector: {}
policyTypes:
- Egress
egress:
- ports:
- port: 53
protocol: UDP
to:
- namespaceSelector: {}
```
## Apply NetPol to App
Finally we can apply the NetworkPolicy to out application by appending the name of the predefined policy to the `networkpolicies` array:
``` yaml
apps:
- name: my-secure-app
[...]
networkpolicies:
- deny-everything
- allow-dns
```
## Allow access to/from Heqet Application
Here is a simple way to apply rules containing other heqet apps. Lets say we have two apps:
``` yaml
apps:
- name: app-one
[...]
networkpolicies:
- deny-app-two
- name: app-two
[...]
```
So we want to deny access from `app-one` to `app-two`. In this case we can use a label that's automatically applied by heqet to every application namespace: `app.heqet.gnu.one/name`
Our policy could look something like this:
``` yaml
networkpolicies:
deny-app-two:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
app.heqet.gnu.one/app: app-two
```

View File

@ -0,0 +1,23 @@
config:
## default: name of this directory
#name:
description: My First Great Project
## NetworkPolicies
# networkPolicy:
# groups: []
# rules: []
apps:
- name: argocd
repoURL: https://argoproj.github.io/argo-helm
chart: argo-cd
targetRevision: 3.17.6
syncWave: "0"
## VaultSecrets:
# secrets:
# - name: argocd-secret
# keys:
# - admin.password

View File

@ -14,14 +14,14 @@ spec:
namespace: heqet
server: 'https://kubernetes.default.svc'
source:
path: .
path: charts/heqet
repoURL: 'https://github.com/nold360/heqet'
targetRevision: k3s
targetRevision: master
helm:
valueFiles:
- values.yaml
project: default
syncPolicy:
automated:
prune: true
selfHeal: true
# syncPolicy:
# automated:
# prune: true
# selfHeal: true

View File

@ -1,50 +0,0 @@
{{/* Generate VaultSecrets */}}
{{- define "gen.secrets" -}}
{{- $context := . }}
{{- with $context }}
{{- range .secrets }}
---
apiVersion: ricoberger.de/v1alpha1
kind: VaultSecret
metadata:
name: {{ .name }}
namespace: {{ $context.namespace | default $context.existingNamespace | default $context.name | quote }}
labels:
app: {{ $context.name }}
annotations:
argocd.argoproj.io/sync-wave: "-1"
spec:
keys:
{{ toYaml .keys | indent 2}}
path: heqet/{{ .fromApp | default $context.name }}/{{ .name }}
type: {{ $context.type | default "Opaque" }}
{{ end }}
{{ end }}
{{- end }}
{{/* Generate NetworkPolicies */}}
{{- define "gen.netpol" -}}
{{- $context := .app }}
{{- $netpols := .netpols }}
{{- with $context }}
{{- with $netpols }}
{{- range $context.networkpolicies }}
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: {{ . }}
namespace: {{ $context.namespace | default $context.existingNamespace | default $context.name | quote }}
labels:
app.heqet.gnu.one/name: {{ $context.name }}
annotations:
argocd.argoproj.io/sync-wave: "-1"
spec:
{{- if not (hasKey $netpols .) }}
{{ fail "ERROR: A NetworkPolicy could not be found in defined Values.networkpolicies!" }}
{{- end }}
{{ get $netpols . | toYaml | indent 2 }}
{{- end }}
{{- end }}
{{ end }}
{{- end }}

View File

@ -1,6 +0,0 @@
{{- if .Values.installCRDs }}
{{- range $path, $_ := .Files.Glob "crds/*.y*ml" }}
{{ $.Files.Get $path }}
---
{{- end }}
{{- end }}

View File

@ -1,66 +0,0 @@
{{- range .Values.apps -}}
{{- if not .disabled }}
{{- if not .existingNamespace }}
---
apiVersion: v1
kind: Namespace
metadata:
name: {{ .namespace | default .name | quote }}
labels:
app.heqet.gnu.one/name: {{ .name }}
{{ with .labels }}{{ toYaml .labels | indent 4}}{{ end }}
annotations:
argocd.argoproj.io/sync-wave: "-42"
{{ with .annotations }}{{ toYaml .annotations | indent 4}}{{ end }}
{{- end }}
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: {{ .name | quote }}
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
labels:
app.heqet.gnu.one/name: {{ .name }}
app.kubernetes.io/name: {{ .name }}
app.kubernetes.io/part-of: heqet
{{ with .labels }}{{ toYaml .labels | indent 4}}{{ end }}
annotations:
argocd.argoproj.io/sync-wave: {{ .syncWave | default "0" | quote }}
{{ with .annotations }}{{ toYaml .annotations | indent 4}}{{ end }}
spec:
project: {{ .project | default $.Values.defaults.project | default "heqet" }}
destination:
namespace: {{ .namespace | default .existingNamespace | default .name | quote }}
server: {{ .server | default $.Values.defaults.server | default "https://kubernetes.default.svc" }}
source:
path: {{ .path | default "" | quote }}
repoURL: {{ .repoURL | default $.Values.defaults.repoURL | quote }}
targetRevision: {{ .targetRevision | default $.Values.defaults.targetRevision | default "HEAD" | quote }}
{{ if .chart }}chart: {{ .chart | quote }}{{ end }}
helm:
{{- with .parameters }}
parameters:
{{ toYaml . | indent 8 }}
{{- end }}
{{- $values := $.Files.Get (printf "values.d/%s.yaml" .name ) | fromYaml | default dict }}
{{- with $values }}
values: |
{{ toYaml $values | indent 8 }}
{{- end }}
syncPolicy:
automated:
prune: {{ .prune | default $.Values.defaults.automated.prune | default "false" }}
selfHeal: {{ .selfHeal | default $.Values.defaults.automated.selfHeal | default "false" }}
{{- if .ignoreDiff }}
ignoreDifferences:
{{ .ignoreDiff | toYaml | indent 4 }}
{{- end }}
{{- if $.Values.generators.vault }}{{- include "gen.secrets" . }}{{- end }}
{{- if hasKey . "networkpolicies" }}
{{- $data := dict "app" . "netpols" $.Values.networkpolicies }}
{{- if $.Values.generators.networkpolicy }}{{- include "gen.netpol" $data }}{{- end }}
{{- end }}
{{- end }}
{{- end }}

File diff suppressed because it is too large Load Diff

View File

@ -1,143 +0,0 @@
replicaCount: 1
deploymentStrategy: {}
image:
repository: ricoberger/vault-secrets-operator
tag: 1.14.2
pullPolicy: IfNotPresent
volumeMounts: []
# - name: ca
# mountPath: "/etc/vault-secrets-operator"
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
environmentVars: []
# Set environment variables from a secret. This must be done, if you use the
# Token or AppRole Auth Methods of Vault.
# Token auth method:
# - name: VAULT_TOKEN
# valueFrom:
# secretKeyRef:
# name: vault-secrets-operator
# key: VAULT_TOKEN
# - name: VAULT_TOKEN_LEASE_DURATION
# value: "300"
# - name: VAULT_CACERT
# value: "/etc/vault-secrets-operator/ca.pem"
# - name: VAULT_TOKEN_RENEWAL_INTERVAL
# value: "43200"
# - name: VAULT_TOKEN_RENEWAL_RETRY_INTERVAL
# value: "30"
# AppRole auth method:
# - name: VAULT_ROLE_ID
# valueFrom:
# secretKeyRef:
# name: vault-secrets-operator
# key: VAULT_ROLE_ID
# - name: VAULT_SECRET_ID
# valueFrom:
# secretKeyRef:
# name: vault-secrets-operator
# key: VAULT_SECRET_ID
# - name: VAULT_TOKEN_RENEWAL_RETRY_INTERVAL
# value: "30"
# - name: VAULT_TOKEN_MAX_TTL
# value: "43200"
# Set the address for vault (by default we assume you are running a dev
# instance of vault in the same namespace as the operator) and specify the
# authentication method for the operator. Possible values are 'token',
# 'kubernetes', or 'approle'.
# If the authentication method is 'kubernetes' the Helm chart
# ensures that the Service Account included the needed rights. The default path
# for the Kubernets Auth method is 'auth/kubernetes', if you enabled it under
# another path you must change the 'kubernetesPath' value. You must also
# provide the role which should be used for the authentication.
#
# If the auth method is 'token' you can specify the 'tokenPath' to read the
# Vault token from a mounted volume instead of an environment variable.
#
# If the auth method is 'approle' you must specify a path for the AppRole Auth
# method with 'appRolePath', by default it is 'auth/approle'. Also
# 'VAULT_ROLE_ID' and 'VAULT_SECRET_ID' must be set for this auth method. With
# this method, the renewal interval is set by default to a half of the token
# lease duration (can be overwritten with 'VAULT_TOKEN_RENEWAL_INTERVAL'), the
# token maximum TTL is set by default to 1382400 seconds (16 days, can be
# overwritten with 'VAULT_TOKEN_MAX_TTL').
#
# The reconciliationTime value determines after which time the Vault secret is
# processed again. This can be used to update a the Kubernetes secret, when the
# Vault secret changes. A value of 0 will disable the automatic update.
# You can specify all namespaces the operator should watch. Therefore pass a
# comma separated list via the namespaces value. If the value is empty the operator
# will watch all namespaces. If the value is empty and rbac.namespaced is set to
# true, then the namespace of the release will be used.
vault:
address: "http://vault.vault.svc.cluster.local:8200"
authMethod: kubernetes
tokenPath: ""
kubernetesPath: auth/kubernetes
kubernetesRole: heqet-app
appRolePath: auth/approle
reconciliationTime: 0
namespaces: ""
crd:
create: false
rbac:
create: true
createrole: true
namespaced: false
serviceAccount:
create: true
name: vault-secrets-operator
# Annotations for vault-secrets-operator pod(s).
podAnnotations: {}
# Additional labels for the vault-secrets-operator pod(s).
podLabels: {}