Argo Workflow 4
Argo Workflow (4)
1 Secrets
Argo Workflow supports the use of Kubernetes Secrets within workflows. A Secret is an object used to store sensitive information, such as passwords, OAuth tokens, etc. Secrets can be stored in Etcd in plain text or base64 encoded form. When using a Secret in a workflow, you need to define a secrets field in the workflow spec, and then reference it in the workflow template using {{workflow.spec.secrets}}.
First, create a secret, and then use it in the workflow.
kubectl create secret generic my-secret --from-literal=mypassword=S00perS3cretPa55word -n argo
# To run this example, first create the secret by running:
# kubectl create secret generic my-secret --from-literal=mypassword=S00perS3cretPa55word
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: secret-example-
spec:
entrypoint: argosay
# To access secrets as files, add a volume entry in spec.volumes[] and
# then in the container template spec, add a mount using volumeMounts.
volumes:
- name: my-secret-vol
secret:
secretName: my-secret # name of an existing k8s secret
templates:
- name: argosay
container:
image: alpine:3.7
command: [sh, -c]
args: ['
echo "secret from env: $MYSECRETPASSWORD";
echo "secret from file: `cat /secret/mountpath/mypassword`"
']
# To access secrets as environment variables, use the k8s valueFrom and
# secretKeyRef constructs.
env:
- name: MYSECRETPASSWORD # name of env var
valueFrom:
secretKeyRef:
name: my-secret # name of an existing k8s secret
key: mypassword # 'key' subcomponent of the secret
volumeMounts:
- name: my-secret-vol # mount file containing secret at /secret/mountpath
mountPath: "/secret/mountpath"
2 Volumes
The following example will dynamically create a volume, and then use this volume in a two-step workflow.
PV is a persistent volume, PVC is a persistent volume claim, PV and PVC are two resource objects in k8s. PV is the actual object of a persistent volume, PVC is the declaration object of a persistent volume. PV is a cluster-level resource, PVC is a namespace-level resource. PV is the actual object of a persistent volume, PVC is the declaration object of a persistent volume. PV is a cluster-level resource, PVC is a namespace-level resource.
This PVC is temporary and will be deleted after the workflow ends.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: volumes-pvc-
spec:
entrypoint: volumes-pvc-example
volumeClaimTemplates: # define volume, same syntax as k8s Pod spec
- metadata:
name: workdir # name of volume claim
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi # Gi => 1024 * 1024 * 1024
templates:
- name: volumes-pvc-example
steps:
- - name: generate
template: argosay
- - name: print
template: print-message
- name: argosay
container:
image: yky8/argosay:v2
command: [sh, -c]
args: ["echo generating message in volume; cowsay hello world | tee /mnt/vol/hello_world.txt"]
# Mount workdir volume at /mnt/vol before invoking docker/whalesay
volumeMounts: # same syntax as k8s Pod spec
- name: workdir
mountPath: /mnt/vol
- name: print-message
container:
image: alpine:latest
command: [sh, -c]
args: ["echo getting message from volume; find /mnt/vol; cat /mnt/vol/hello_world.txt"]
# Mount workdir volume at /mnt/vol before invoking docker/whalesay
volumeMounts: # same syntax as k8s Pod spec
- name: workdir
mountPath: /mnt/vol
Volume is a very effective way to move large amounts of data from one step of a workflow to another. Depending on the system, some volumes may be accessed from multiple steps at the same time.
Sometimes, we want to get an existing volume, rather than dynamically creating a new one:
# Define Kubernetes PVC
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: my-existing-volume
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: volumes-existing-
spec:
entrypoint: volumes-existing-example
volumes:
# Pass my-existing-volume as an argument to the volumes-existing-example template
# Same syntax as k8s Pod spec
- name: workdir
persistentVolumeClaim:
claimName: my-existing-volume
templates:
- name: volumes-existing-example
steps:
- - name: generate
template: argosay
- - name: print
template: print-message
- name: argosay
container:
image: yky8/argosay:v2
command: [sh, -c]
args: ["echo generating message in volume; cowsay hello world | tee /mnt/vol/hello_world.txt"]
volumeMounts:
- name: workdir
mountPath: /mnt/vol
- name: print-message
container:
image: alpine:latest
command: [sh, -c]
args: ["echo getting message from volume; find /mnt/vol; cat /mnt/vol/hello_world.txt"]
volumeMounts:
- name: workdir
mountPath: /mnt/vol
You can also define a volume under the resource step in the workflow’s template field, so you can define the volume under the spec field of the workflow, and then use this volume in the workflow’s template.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: template-level-volume-
spec:
entrypoint: generate-and-use-volume
templates:
- name: generate-and-use-volume
steps:
- - name: generate-volume
template: generate-volume
arguments:
parameters:
- name: pvc-size
# In a real-world example, this could be generated by a previous workflow step.
value: '1Gi'
- - name: generate
template: argosay
arguments:
parameters:
- name: pvc-name
value: '{{steps.generate-volume.outputs.parameters.pvc-name}}'
- - name: print
template: print-message
arguments:
parameters:
- name: pvc-name
value: '{{steps.generate-volume.outputs.parameters.pvc-name}}'
- name: generate-volume
inputs:
parameters:
- name: pvc-size
resource:
action: create
setOwnerReference: true
manifest: |
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
generateName: pvc-example-
spec:
accessModes: ['ReadWriteOnce', 'ReadOnlyMany']
resources:
requests:
storage: '{{inputs.parameters.pvc-size}}'
outputs:
parameters:
- name: pvc-name
valueFrom:
jsonPath: '{.metadata.name}'
- name: argosay
inputs:
parameters:
- name: pvc-name
volumes:
- name: workdir
persistentVolumeClaim:
claimName: '{{inputs.parameters.pvc-name}}'
container:
image: yky8/argosay:v2
command: [sh, -c]
args: ["echo generating message in volume; cowsay hello world | tee /mnt/vol/hello_world.txt"]
volumeMounts:
- name: workdir
mountPath: /mnt/vol
- name: print-message
inputs:
parameters:
- name: pvc-name
volumes:
- name: workdir
persistentVolumeClaim:
claimName: '{{inputs.parameters.pvc-name}}'
container:
image: alpine:latest
command: [sh, -c]
args: ["echo getting message from volume; find /mnt/vol; cat /mnt/vol/hello_world.txt"]
volumeMounts:
- name: workdir
mountPath: /mnt/vol
To execute this, you need to first ensure that the default service account in the argo namespace has permissions, otherwise an error will occur.
Like me at the beginning, I didn’t have the permission to create PVC, I need to give the default service account permission.
# role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: argo
name: pvc-creator
rules:
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["create", "delete", "get", "list", "watch"]
---
# rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: pvc-creator-binding
namespace: argo
subjects:
- kind: ServiceAccount
name: default
namespace: argo
roleRef:
kind: Role
name: pvc-creator
apiGroup: rbac.authorization.k8s.io
kubectl apply -f role_bind.yaml
argo submit volume-tpl.yaml -n argo
