k8s kubernetes Lesson 2 yaml

https://kubernetes.github.io/ingress-nginx/examples/docker-registry/
https://sueboy.blogspot.com/2019/11/kubernetes-nodeport.html

PS：Close basic auth just remark configmap.yml auth:

 #auth:  
    # htpasswd:  
    #   realm: basic-realm  
    #   path: /auth/htpasswd

README.md

  
# Docker Registry K8s  
  
Domain：docker-registry  
Port:30500  
/etc/hosts  add 192.168.99.118 docker-registry  ## ip is k8s host eth2's ip  
Finish command： docker login -u username -p password docker-registry:30500  
  

[![](https://3.bp.blogspot.com/-Wj82n6nvHkw/XHUVI-gNy7I/AAAAAAAAUE8/rs1l8mQuu8cKvkoMcZGn0lOfJUWY3udMgCLcBGAs/s640/dockerloginsuccess.jpg)](https://3.bp.blogspot.com/-Wj82n6nvHkw/XHUVI-gNy7I/AAAAAAAAUE8/rs1l8mQuu8cKvkoMcZGn0lOfJUWY3udMgCLcBGAs/s1600/dockerloginsuccess.jpg)

https://docs.docker.com/registry/insecure/  

Deploy a plain HTTP registry
----------------------------

```bash  
/etc/docker/daemon.json  
  
{  
  "insecure-registries" : ["docker-registry:30500"]  
}  
  
Restart Docker for the changes to take effect

https://blog.csdn.net/zsd498537806/article/details/79290732
https://serverfault.com/questions/611120/failed-tls-handshake-does-not-contain-any-ip-sans
http://dockone.io/article/684

#-subj “/C=/ST=/L=/O=/OU=/CN=192.168.99.118”
-subj “/C=/ST=/L=/O=/OU=/CN=docker-registry”

Generate private key & certificate

openssl req -nodes -newkey rsa:4096 -keyout certs/domain.key -out certs/domain.csr -subj "/C=/ST=/L=/O=/OU=/CN=docker-registry"  
openssl x509 -req -sha256 -days 365 -in certs/domain.csr -signkey certs/domain.key -out certs/domain.crt  
  
#> openssl req \  
#     -newkey rsa:4096 -nodes -keyout certs/domain.key \  
#     -x509 -days 365 -out certs/domain.crt

docker run --rm --entrypoint htpasswd registry:2 -Bbn username password > htpasswd

K8s Secret

cat domain.crt | base64  
cat domain.key | base64

No bash64, just online

domain.crt base64

LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVzRENDQXBnQ0NRRHpteitOQUZCMGRUQU5C  
Z2txaGtpRzl3MEJBUXNGQURBYU1SZ3dGZ1lEVlFRRERBOWsKYjJOclpYSXRjbVZuYVhOMGNua3dI  
aGNOTVRrd01qSTJNVEF3TlRBMldoY05NakF3TWpJMk1UQXdOVEEyV2pBYQpNUmd3RmdZRFZRUURE  
QTlrYjJOclpYSXRjbVZuYVhOMGNua3dnZ0lpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElDCkR3QXdn  
Z0lLQW9JQ0FRRFBKcHhZK0RVQ05aMVh4Tkc0bUJlaFp2ZXZ4R21XMFdZandhRklzcFQ0L09ETWgw  
ZVgKRXRmT0xpaUZibXFzMi93L01CeWlUSWlZczltMFdhdG5uVDc5bm5wZUVHdFNtQUJHeXMyU1hx  
dDZ3SzJubHJSLwowVllaVEpJQ2t2amE5WExtQ2w2eEgrbFFxWVFnNHlHSWxuZ2ZXUFBCTzkvVFhn  
S2Jmc2xBTk9uTXY0ckVTMlNFCkZLQzNuSTRqY1JWc1BQSUNkNDNJVlU1akNiRllublZJS2l2eTdq  
Y0l0Q2JoelZsTHRuOHFIRE5YcElHSlpTTlYKUkdSdUVhN3ZFbUloei90blJZdFJRTmpSNGhQNkM5  
eG1QMFpBZFhjQWZYMTk0aVVKNDRFYnZUZ0NKK0I4Yi9jdwpEZ29IRWxCbGtJaUtmSm9wL0RhR3B5  
OUY0VjRhbHZIcnNMdFdYYTZYWmQzeGxmWWx6S2xMZ0IzTDJVQ2VCUWljCk0yUDlXMEZZZGFIR3Nj  
UkFyNGQvbEtiZE1PYXNDV2dXaHlGekRDMUFYQkcxdU9SSXFBbi84OTJPQnk1UVE2SDQKSk9yOXNR  
RGhRejVEdWZReFlGNDZUZjZUNk9MOGp6TkUrUk11YmR2RmVVS1ZwYzJXazgrVWY3UE9OdW9iRC9X  
ZgpTRFVCYjNsSCtGVTFZalVhR0Y5VzNwSE14NXZzTU8xOFJ3UTIzU2ZFT1dMZFFseU8rYXg3STZK  
Vm5CbG1xaStFCk16QjFHMzVTbmlMMWcyVDRkbFlIWThIbllaU1A3RFdYR3JrZ2l6SloweUUveE55  
dkJuLy9GeWtIYTlIdjBhUk0KNE45OUhHRkthTmNzdkFDR2NSU2QvYUdHUDdqWHNEb3R2R1U5VjU4  
ems2S21oVGRjN3V0cE9Hb0Nud0lEQVFBQgpNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUNBUUM4U1or  
N0V3U0liY0tDT0xoV3NvbDEzNUVIYVdiRTEzOFRRdHBVCk55aExqZXM3THVQSjg3YVdjK0NWT3VM  
WEdIY2FtTiszQWhKL1k3L1U5NlVCWS8yTDREUnd6b0RDYzhhQ3ZQY0oKZDhWcGxvUXJuRzRueUgx  
VTJUV243SXo1UDNrYjVZeE82UHlVdlNZMTJTaXBxWmVuQU1STzVQSDRlUldwbDkwbAp3cHJDVU1T  
NGhaL3FGVE1ORG1HSzBGS243VHZRekZrSVo1eEhUK0VncTNzcERVY1BCenV4WVdGTG9pT2ZGUFRZ  
CnZkSDc5Vis5K1p6T3ZBVlF3ZHhHSVlOaGlpNGd1OTg4N3Jld3RKbEZUQ1pXT05DWnNMcUw0RHBt  
WmJhbUlxRi8KakF5ZUR4T2kwN2VKcTlUREFLUlErZzg1NzMvR2JhVU5MVHVndzM0Z1pNZUttUnNz  
UjBEZWc5ckFHeXRRM2pqRApqeWJLYmo1SE8vT25xNTlGSk9rRi94eklZYWtXcjVPem1kRzJDdVhN  
OXhSN3BYSm5IM0RKcWUwc2FQd2ZMV2xPClF3L1RRM1lpMHg4ajhwOTRpeHZWbnRPRFRIUXBmeUFD  
MDloaFNmVG1sMnJXT3FqTzJ4d1dzUkhEOXBUQ0FpMG0KTHlZQS9UTVVXRWdMdklmMEZjOXRZZGkr  
ck9HQVJuRWdXdjdhRjhKL0pRUlZ2ZjBVL2hGTmo4bFRmY0RuOU95WQprVEtWdHNRVllqZXNBU1lI  
dzYyRmZoUkRIZmZRbUlscTN3emljU3MvbnBQMzZraExCZGt0TnZzakVNaE40a0JLCjR5NlJWekdZ  
R20rakpVQk8zV3ZQbHF0T0pwVUFCbHVCWFZOK3hhMnZOODNGYjZtMFlJTmVWbXNFQmxnVTVGT1oK  
a2xFVTJ3PT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=

domain.key base64

LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUpRd0lCQURBTkJna3Foa2lHOXcwQkFRRUZB  
QVNDQ1Mwd2dna3BBZ0VBQW9JQ0FRRFBKcHhZK0RVQ05aMVgKeE5HNG1CZWhadmV2eEdtVzBXWWp3  
YUZJc3BUNC9PRE1oMGVYRXRmT0xpaUZibXFzMi93L01CeWlUSWlZczltMApXYXRublQ3OW5ucGVF  
R3RTbUFCR3lzMlNYcXQ2d0sybmxyUi8wVllaVEpJQ2t2amE5WExtQ2w2eEgrbFFxWVFnCjR5R0ls  
bmdmV1BQQk85L1RYZ0tiZnNsQU5Pbk12NHJFUzJTRUZLQzNuSTRqY1JWc1BQSUNkNDNJVlU1akNi  
RlkKbm5WSUtpdnk3amNJdENiaHpWbEx0bjhxSEROWHBJR0paU05WUkdSdUVhN3ZFbUloei90blJZ  
dFJRTmpSNGhQNgpDOXhtUDBaQWRYY0FmWDE5NGlVSjQ0RWJ2VGdDSitCOGIvY3dEZ29IRWxCbGtJ  
aUtmSm9wL0RhR3B5OUY0VjRhCmx2SHJzTHRXWGE2WFpkM3hsZllsektsTGdCM0wyVUNlQlFpY00y  
UDlXMEZZZGFIR3NjUkFyNGQvbEtiZE1PYXMKQ1dnV2h5RnpEQzFBWEJHMXVPUklxQW4vODkyT0J5  
NVFRNkg0Sk9yOXNRRGhRejVEdWZReFlGNDZUZjZUNk9MOApqek5FK1JNdWJkdkZlVUtWcGMyV2s4  
K1VmN1BPTnVvYkQvV2ZTRFVCYjNsSCtGVTFZalVhR0Y5VzNwSE14NXZzCk1PMThSd1EyM1NmRU9X  
TGRRbHlPK2F4N0k2SlZuQmxtcWkrRU16QjFHMzVTbmlMMWcyVDRkbFlIWThIbllaU1AKN0RXWEdy  
a2dpekpaMHlFL3hOeXZCbi8vRnlrSGE5SHYwYVJNNE45OUhHRkthTmNzdkFDR2NSU2QvYUdHUDdq  
WApzRG90dkdVOVY1OHprNkttaFRkYzd1dHBPR29DbndJREFRQUJBb0lDQVFDb0FkZXliUFNKS3pU  
R3Q1R2tsbTRyCjhKZ1lUVU9hRjVFd2xOMmJXWU5yZ2FWTkhMYmFkMlRpQ3h2QVJISnVhaWpta3Iz  
V2dCNFZRMkRoWGd5Zmg3Qk0KQUZtY3F0SndkV2xjQVNYRGZkYTBRMHRVYmUxTThWcFVRSDczYVg0  
K0QrNDFWdG0vN0hMTmdmN3dzV1pFSWsyZwpsd056aUgwYkw5Y0xpRVB2T0RzYlhZcE5kQnc4NTRJ  
R0NRcGJBbDl1bnV0bFJ0T09WR216OTJ1SHhWUzN4OFJIClRoOENoeUkxNXRkeWlid1BTam4xN2kx  
Rll1cktRQ0w0Qm8yTHhxd2lZazJ6VDNzVDFuVkRuSW01Tmg5MktnRnYKVEJYTkp2bzl1OGx3Z0x6  
NDQzbG1mak1hbWJDd24rSlFJL0Zpdmx1UGZrbUduZEVYV3NjRGRTcGVoR2dIVVJwTwpGWURKU29j  
aTlGVG5NRTF4a3c4TUR5V1JjSzdvSjJrRzd0RXh4aDBML1NjK1dmY2RKVFJyRW9BR3VxNzNYdWZG  
Ck1CdVRoK3JCVU9ITVpSTVJMTXN6aGVTekp1SXBZU1BsNUF1VUY5cHVNd21vYkFpbXBzdmpQVEpr  
RlR3bytLdmQKYUVlNW5IMFlmTlRNS3JSU1YybjBaTmp0dHB3VU9VZ2lXVE9VdnJXOGNiMlVJZjRp  
RVk3VjJBc3ArKzVvK25QdQpZRkdiZWd0d1U3dURyNTN2YjNPRE9ZRVUrUlZmYTlScWxXMUswYlpm  
eGZBcCtzdVNQTHczclBiNk4xZ0UybWhFCnNUcy9FYmQ5dW5Vdm1XZ0tzdGtBeDBuRVM4Z1VqNW9Z  
c1F5Um1wM1NaaUJXUmlXYnZoSzRaQ3k0V3NaZUZWZEoKMEtuNDZCeXB1Sndtelozd2hRMDBZUUtD  
QVFFQS9XODlsa0c1eW9UVXlEZnI3M0hqMHNyZ0o3amlLdlNoZ29PVwpCck1ndkxxeEMzUVRaZjg2  
VHpObnhoVDFwSS9sUnNRN2tiYjJhclpHZ3phRHlNemV2QXpYMUJiNnRkakxnVVIvCi9ZMTR1QW9V  
VzhYNEhrY0Rhb2xtVkFjOXdsRmpJcHVjY25YdEZJZ3NacnVydEJSQW9mSU91aUV4bzJCQkc3aUEK  
MW1qN2RqZlZwSTQzWWd4UHBiZndmVjV0bC9CTENMQis0RzZkclhvaExBdTlqSWlKNWw4MEEydG1q  
R3Z3S3lVTApHc3RGamc1MVcxTjQyTC8rY0FSQ213a3pUUTkwMFZaS1BaOUdtZTE2NndZdjl4eXds  
ZjN6L09udGJsZjRLaXE4ClVVQXR3RFF3bWFwN3o1OUhpckp1TEczMDk3QWRlNTZjQ1JOMk53c0Yz  
YTJQM09VeFVRS0NBUUVBMFQ5dHlzK24KVHdkd0JNK0lRcVBQUWt0ZjhodENSUW1Wamw0WWVZc1pt  
R0F2TXRqN00wYVhjcnVLT0k2VjBmc0k2T0RrbWRDSQpnZWYwUk15WXR3czdMcGtXWmIyQzlFdzk0  
K3VpL2tEQVlxUUJXczBVaTQwbTdNeDgwN2pKdGZONVJad3d1MGxNCm1kR1ZSa3hkbWVtM0V6Mmhs  
R2ltZUMvcnB5REtiMkY1SU9Mdjk1cUdlWHBlOGNwUDhVMDd6dTdqdmdDNVRSeDcKV0pYYVVFSEt4  
bjVHVHU2UUluMXQrZmw5MnRnNXprNmNjaklUeStvUG4yNWU1ZnR3ZlNaVzg1Tno2dFQzRE04bwpM  
Z0h4aFBNS0RzcjZySnVRSExObHI1MnY4TDlwVlZlWlpOanlmbDNxUmhqOE5zanN4OEhCczV1Z1ZF  
YU9MWkJTClFrU1pCYzNkYldONDd3S0NBUUFMWVhaemkrb0s5U0lWQjZYWnI5U2FQN0FIT0o4M2JK  
cmdJSWFaTUd3VTdaRkwKd1YrcG9lZDliUUtJKzZINFRObnY3cG42Tml3a3FLREwvYU8zWGJQVzRB  
REZMbTc5d1ltSitZaUIyMXVEY0RodgowcGtNeXdXVlFDeFBNN0FMVE5wVjBJVWJBcjZ0d0Z1MkZW  
RmxtYzg1Tm9TQXlVTFFZbDhxSG9QeW5PNUY1Y1g3ClNNT2w3SVdiSElKNUxzZGZIbFJRUkpTK0FW  
bWxVNm12cjZ0SVowMUNBaDUwMU5yYkRYSFFqaTVNZXJwWjRiNVYKWm84enRKTjVka2VEYk10WDk0  
eTRaZGFabzlCbzB5aEU2NWMxMHB6L2IxTUlsWHlyQ3RwREVuNDRhOWhpSERYZQpnTEJrcDNOd0FB  
bEFoNE9McCtYN0NpbFpvdjBaQmE2eXdMdUlJT2F4QW9JQkFILytaSkppWXQwY2w1RFp3V2ZCCis4  
MTVmYjhDblk3TlJnWkgySURBYlcrZnBqOWNLb3c2NmZqc2JxZ1p6OXNaeTJGbjcyMkcrWGs5QVZY  
ME1KWUcKOExTWVAvTUw4VWF1UEUxbW5kVVlscFcvN0hRMmJoTE96TGljZk42bGNmMHlhVVFEUG12  
UmNtSEVxYUxPbzVLNgpPNW94WU56L0JwVU4rdi9JTnd2YWU5VjUrMzlocmxBd1lsTlVoWUg4MUZE  
SUg5YmhhbmVZckhxTGIzY1lSNGtUClJMNlRmdWNhQW1PZUVUWUVmWmVNem5XcjNDQW5sRmJabElM  
cFF1M0t2ZlZRaE45b1ZaWW9RVk9qTExFK1dUQ2oKU0tLOHBYSm82aEo2U2x6aE9QVlRHQ2x2NnlV  
UkhYU3pZOU8vSTljczFlZWZrbnI2alkyaGNpL0c5ak51c2J0dgoxVkVDZ2dFQkFJVUdEbEcxQmpP  
dTVhcVZ1eDZzTzUwbHB4eVpienFvMm1TRVMwZHNYNDcvclN6ZGRoblE4QnVVCjlURHF3aDBlM0pZ  
dGlUczM4Z21RT1pVdndEM2FnNjdDU0t1T3FNUWM3aU00S01rbmdrakxOMnhIM2luYjZxUjMKc0ZL  
YVluWDlMdTBZeVVZU013cGwyNjM4ZHoxanY3a05wRE8xd2p0bTh5TjdxQk00d3IrU3JNV0EyWHlh  
SG1hZgpBZE5hNzdxdHppL0JIcXdVTFZsbDFTSitIU1dqdHU2QmRNQTlLandzMllieWxUTm1UbmZW  
TzdLeVMzTFAwY2xCCm1MaEVhYS93d2J1Mm5sTWRxdDJpeTQzcE5QYmhRZ0lOck95dDR6cXhNZEVX  
dWtSeHAwSGFieXdFcldFWXJFQ1cKdS90ZDhKZ291QmZqZ0JDQ3hnNjNvWlJVS1ZnU1FQST0KLS0t  
LS1FTkQgUFJJVkFURSBLRVktLS0tLQo=

htpasswd base64

dXNlcm5hbWU6JDJ5JDA1JGprbU5CSm1DVTYyV0pKc0tqcWx3RC5BQmpLcUswc2R0SU9lZ29kb3VrQzFpR0owc05ENjhD

K8s Namespace namespace.json

{  
  "kind": "Namespace",  
  "apiVersion": "v1",  
  "metadata": {  
    "name": "docker-registry",  
    "labels": {  
      "name": "test-dev"  
    }  
  }  
}

K8s Secret secret.yml

apiVersion: v1  
kind: Secret  
metadata:  
  name: registry-tls  
  namespace: docker-registry  
type: Opaque  
data:  
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVzRENDQXBnQ0NRRHpteitOQUZCMGRUQU5CZ2txaGtpRzl3MEJBUXNGQURBYU1SZ3dGZ1lEVlFRRERBOWsKYjJOclpYSXRjbVZuYVhOMGNua3dIaGNOTVRrd01qSTJNVEF3TlRBMldoY05NakF3TWpJMk1UQXdOVEEyV2pBYQpNUmd3RmdZRFZRUUREQTlrYjJOclpYSXRjbVZuYVhOMGNua3dnZ0lpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElDCkR3QXdnZ0lLQW9JQ0FRRFBKcHhZK0RVQ05aMVh4Tkc0bUJlaFp2ZXZ4R21XMFdZandhRklzcFQ0L09ETWgwZVgKRXRmT0xpaUZibXFzMi93L01CeWlUSWlZczltMFdhdG5uVDc5bm5wZUVHdFNtQUJHeXMyU1hxdDZ3SzJubHJSLwowVllaVEpJQ2t2amE5WExtQ2w2eEgrbFFxWVFnNHlHSWxuZ2ZXUFBCTzkvVFhnS2Jmc2xBTk9uTXY0ckVTMlNFCkZLQzNuSTRqY1JWc1BQSUNkNDNJVlU1akNiRllublZJS2l2eTdqY0l0Q2JoelZsTHRuOHFIRE5YcElHSlpTTlYKUkdSdUVhN3ZFbUloei90blJZdFJRTmpSNGhQNkM5eG1QMFpBZFhjQWZYMTk0aVVKNDRFYnZUZ0NKK0I4Yi9jdwpEZ29IRWxCbGtJaUtmSm9wL0RhR3B5OUY0VjRhbHZIcnNMdFdYYTZYWmQzeGxmWWx6S2xMZ0IzTDJVQ2VCUWljCk0yUDlXMEZZZGFIR3NjUkFyNGQvbEtiZE1PYXNDV2dXaHlGekRDMUFYQkcxdU9SSXFBbi84OTJPQnk1UVE2SDQKSk9yOXNRRGhRejVEdWZReFlGNDZUZjZUNk9MOGp6TkUrUk11YmR2RmVVS1ZwYzJXazgrVWY3UE9OdW9iRC9XZgpTRFVCYjNsSCtGVTFZalVhR0Y5VzNwSE14NXZzTU8xOFJ3UTIzU2ZFT1dMZFFseU8rYXg3STZKVm5CbG1xaStFCk16QjFHMzVTbmlMMWcyVDRkbFlIWThIbllaU1A3RFdYR3JrZ2l6SloweUUveE55dkJuLy9GeWtIYTlIdjBhUk0KNE45OUhHRkthTmNzdkFDR2NSU2QvYUdHUDdqWHNEb3R2R1U5VjU4ems2S21oVGRjN3V0cE9Hb0Nud0lEQVFBQgpNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUNBUUM4U1orN0V3U0liY0tDT0xoV3NvbDEzNUVIYVdiRTEzOFRRdHBVCk55aExqZXM3THVQSjg3YVdjK0NWT3VMWEdIY2FtTiszQWhKL1k3L1U5NlVCWS8yTDREUnd6b0RDYzhhQ3ZQY0oKZDhWcGxvUXJuRzRueUgxVTJUV243SXo1UDNrYjVZeE82UHlVdlNZMTJTaXBxWmVuQU1STzVQSDRlUldwbDkwbAp3cHJDVU1TNGhaL3FGVE1ORG1HSzBGS243VHZRekZrSVo1eEhUK0VncTNzcERVY1BCenV4WVdGTG9pT2ZGUFRZCnZkSDc5Vis5K1p6T3ZBVlF3ZHhHSVlOaGlpNGd1OTg4N3Jld3RKbEZUQ1pXT05DWnNMcUw0RHBtWmJhbUlxRi8KakF5ZUR4T2kwN2VKcTlUREFLUlErZzg1NzMvR2JhVU5MVHVndzM0Z1pNZUttUnNzUjBEZWc5ckFHeXRRM2pqRApqeWJLYmo1SE8vT25xNTlGSk9rRi94eklZYWtXcjVPem1kRzJDdVhNOXhSN3BYSm5IM0RKcWUwc2FQd2ZMV2xPClF3L1RRM1lpMHg4ajhwOTRpeHZWbnRPRFRIUXBmeUFDMDloaFNmVG1sMnJXT3FqTzJ4d1dzUkhEOXBUQ0FpMG0KTHlZQS9UTVVXRWdMdklmMEZjOXRZZGkrck9HQVJuRWdXdjdhRjhKL0pRUlZ2ZjBVL2hGTmo4bFRmY0RuOU95WQprVEtWdHNRVllqZXNBU1lIdzYyRmZoUkRIZmZRbUlscTN3emljU3MvbnBQMzZraExCZGt0TnZzakVNaE40a0JLCjR5NlJWekdZR20rakpVQk8zV3ZQbHF0T0pwVUFCbHVCWFZOK3hhMnZOODNGYjZtMFlJTmVWbXNFQmxnVTVGT1oKa2xFVTJ3PT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQ==  
  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUpRd0lCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQ1Mwd2dna3BBZ0VBQW9JQ0FRRFBKcHhZK0RVQ05aMVgKeE5HNG1CZWhadmV2eEdtVzBXWWp3YUZJc3BUNC9PRE1oMGVYRXRmT0xpaUZibXFzMi93L01CeWlUSWlZczltMApXYXRublQ3OW5ucGVFR3RTbUFCR3lzMlNYcXQ2d0sybmxyUi8wVllaVEpJQ2t2amE5WExtQ2w2eEgrbFFxWVFnCjR5R0lsbmdmV1BQQk85L1RYZ0tiZnNsQU5Pbk12NHJFUzJTRUZLQzNuSTRqY1JWc1BQSUNkNDNJVlU1akNiRlkKbm5WSUtpdnk3amNJdENiaHpWbEx0bjhxSEROWHBJR0paU05WUkdSdUVhN3ZFbUloei90blJZdFJRTmpSNGhQNgpDOXhtUDBaQWRYY0FmWDE5NGlVSjQ0RWJ2VGdDSitCOGIvY3dEZ29IRWxCbGtJaUtmSm9wL0RhR3B5OUY0VjRhCmx2SHJzTHRXWGE2WFpkM3hsZllsektsTGdCM0wyVUNlQlFpY00yUDlXMEZZZGFIR3NjUkFyNGQvbEtiZE1PYXMKQ1dnV2h5RnpEQzFBWEJHMXVPUklxQW4vODkyT0J5NVFRNkg0Sk9yOXNRRGhRejVEdWZReFlGNDZUZjZUNk9MOApqek5FK1JNdWJkdkZlVUtWcGMyV2s4K1VmN1BPTnVvYkQvV2ZTRFVCYjNsSCtGVTFZalVhR0Y5VzNwSE14NXZzCk1PMThSd1EyM1NmRU9XTGRRbHlPK2F4N0k2SlZuQmxtcWkrRU16QjFHMzVTbmlMMWcyVDRkbFlIWThIbllaU1AKN0RXWEdya2dpekpaMHlFL3hOeXZCbi8vRnlrSGE5SHYwYVJNNE45OUhHRkthTmNzdkFDR2NSU2QvYUdHUDdqWApzRG90dkdVOVY1OHprNkttaFRkYzd1dHBPR29DbndJREFRQUJBb0lDQVFDb0FkZXliUFNKS3pUR3Q1R2tsbTRyCjhKZ1lUVU9hRjVFd2xOMmJXWU5yZ2FWTkhMYmFkMlRpQ3h2QVJISnVhaWpta3IzV2dCNFZRMkRoWGd5Zmg3Qk0KQUZtY3F0SndkV2xjQVNYRGZkYTBRMHRVYmUxTThWcFVRSDczYVg0K0QrNDFWdG0vN0hMTmdmN3dzV1pFSWsyZwpsd056aUgwYkw5Y0xpRVB2T0RzYlhZcE5kQnc4NTRJR0NRcGJBbDl1bnV0bFJ0T09WR216OTJ1SHhWUzN4OFJIClRoOENoeUkxNXRkeWlid1BTam4xN2kxRll1cktRQ0w0Qm8yTHhxd2lZazJ6VDNzVDFuVkRuSW01Tmg5MktnRnYKVEJYTkp2bzl1OGx3Z0x6NDQzbG1mak1hbWJDd24rSlFJL0Zpdmx1UGZrbUduZEVYV3NjRGRTcGVoR2dIVVJwTwpGWURKU29jaTlGVG5NRTF4a3c4TUR5V1JjSzdvSjJrRzd0RXh4aDBML1NjK1dmY2RKVFJyRW9BR3VxNzNYdWZGCk1CdVRoK3JCVU9ITVpSTVJMTXN6aGVTekp1SXBZU1BsNUF1VUY5cHVNd21vYkFpbXBzdmpQVEprRlR3bytLdmQKYUVlNW5IMFlmTlRNS3JSU1YybjBaTmp0dHB3VU9VZ2lXVE9VdnJXOGNiMlVJZjRpRVk3VjJBc3ArKzVvK25QdQpZRkdiZWd0d1U3dURyNTN2YjNPRE9ZRVUrUlZmYTlScWxXMUswYlpmeGZBcCtzdVNQTHczclBiNk4xZ0UybWhFCnNUcy9FYmQ5dW5Vdm1XZ0tzdGtBeDBuRVM4Z1VqNW9Zc1F5Um1wM1NaaUJXUmlXYnZoSzRaQ3k0V3NaZUZWZEoKMEtuNDZCeXB1Sndtelozd2hRMDBZUUtDQVFFQS9XODlsa0c1eW9UVXlEZnI3M0hqMHNyZ0o3amlLdlNoZ29PVwpCck1ndkxxeEMzUVRaZjg2VHpObnhoVDFwSS9sUnNRN2tiYjJhclpHZ3phRHlNemV2QXpYMUJiNnRkakxnVVIvCi9ZMTR1QW9VVzhYNEhrY0Rhb2xtVkFjOXdsRmpJcHVjY25YdEZJZ3NacnVydEJSQW9mSU91aUV4bzJCQkc3aUEKMW1qN2RqZlZwSTQzWWd4UHBiZndmVjV0bC9CTENMQis0RzZkclhvaExBdTlqSWlKNWw4MEEydG1qR3Z3S3lVTApHc3RGamc1MVcxTjQyTC8rY0FSQ213a3pUUTkwMFZaS1BaOUdtZTE2NndZdjl4eXdsZjN6L09udGJsZjRLaXE4ClVVQXR3RFF3bWFwN3o1OUhpckp1TEczMDk3QWRlNTZjQ1JOMk53c0YzYTJQM09VeFVRS0NBUUVBMFQ5dHlzK24KVHdkd0JNK0lRcVBQUWt0ZjhodENSUW1Wamw0WWVZc1ptR0F2TXRqN00wYVhjcnVLT0k2VjBmc0k2T0RrbWRDSQpnZWYwUk15WXR3czdMcGtXWmIyQzlFdzk0K3VpL2tEQVlxUUJXczBVaTQwbTdNeDgwN2pKdGZONVJad3d1MGxNCm1kR1ZSa3hkbWVtM0V6MmhsR2ltZUMvcnB5REtiMkY1SU9Mdjk1cUdlWHBlOGNwUDhVMDd6dTdqdmdDNVRSeDcKV0pYYVVFSEt4bjVHVHU2UUluMXQrZmw5MnRnNXprNmNjaklUeStvUG4yNWU1ZnR3ZlNaVzg1Tno2dFQzRE04bwpMZ0h4aFBNS0RzcjZySnVRSExObHI1MnY4TDlwVlZlWlpOanlmbDNxUmhqOE5zanN4OEhCczV1Z1ZFYU9MWkJTClFrU1pCYzNkYldONDd3S0NBUUFMWVhaemkrb0s5U0lWQjZYWnI5U2FQN0FIT0o4M2JKcmdJSWFaTUd3VTdaRkwKd1YrcG9lZDliUUtJKzZINFRObnY3cG42Tml3a3FLREwvYU8zWGJQVzRBREZMbTc5d1ltSitZaUIyMXVEY0RodgowcGtNeXdXVlFDeFBNN0FMVE5wVjBJVWJBcjZ0d0Z1MkZWRmxtYzg1Tm9TQXlVTFFZbDhxSG9QeW5PNUY1Y1g3ClNNT2w3SVdiSElKNUxzZGZIbFJRUkpTK0FWbWxVNm12cjZ0SVowMUNBaDUwMU5yYkRYSFFqaTVNZXJwWjRiNVYKWm84enRKTjVka2VEYk10WDk0eTRaZGFabzlCbzB5aEU2NWMxMHB6L2IxTUlsWHlyQ3RwREVuNDRhOWhpSERYZQpnTEJrcDNOd0FBbEFoNE9McCtYN0NpbFpvdjBaQmE2eXdMdUlJT2F4QW9JQkFILytaSkppWXQwY2w1RFp3V2ZCCis4MTVmYjhDblk3TlJnWkgySURBYlcrZnBqOWNLb3c2NmZqc2JxZ1p6OXNaeTJGbjcyMkcrWGs5QVZYME1KWUcKOExTWVAvTUw4VWF1UEUxbW5kVVlscFcvN0hRMmJoTE96TGljZk42bGNmMHlhVVFEUG12UmNtSEVxYUxPbzVLNgpPNW94WU56L0JwVU4rdi9JTnd2YWU5VjUrMzlocmxBd1lsTlVoWUg4MUZESUg5YmhhbmVZckhxTGIzY1lSNGtUClJMNlRmdWNhQW1PZUVUWUVmWmVNem5XcjNDQW5sRmJabElMcFF1M0t2ZlZRaE45b1ZaWW9RVk9qTExFK1dUQ2oKU0tLOHBYSm82aEo2U2x6aE9QVlRHQ2x2NnlVUkhYU3pZOU8vSTljczFlZWZrbnI2alkyaGNpL0c5ak51c2J0dgoxVkVDZ2dFQkFJVUdEbEcxQmpPdTVhcVZ1eDZzTzUwbHB4eVpienFvMm1TRVMwZHNYNDcvclN6ZGRoblE4QnVVCjlURHF3aDBlM0pZdGlUczM4Z21RT1pVdndEM2FnNjdDU0t1T3FNUWM3aU00S01rbmdrakxOMnhIM2luYjZxUjMKc0ZLYVluWDlMdTBZeVVZU013cGwyNjM4ZHoxanY3a05wRE8xd2p0bTh5TjdxQk00d3IrU3JNV0EyWHlhSG1hZgpBZE5hNzdxdHppL0JIcXdVTFZsbDFTSitIU1dqdHU2QmRNQTlLandzMllieWxUTm1UbmZWTzdLeVMzTFAwY2xCCm1MaEVhYS93d2J1Mm5sTWRxdDJpeTQzcE5QYmhRZ0lOck95dDR6cXhNZEVXdWtSeHAwSGFieXdFcldFWXJFQ1cKdS90ZDhKZ291QmZqZ0JDQ3hnNjNvWlJVS1ZnU1FQST0KLS0tLS1FTkQgUFJJVkFURSBLRVktLS0tLQ==  
  HTPASSWD: dXNlcm5hbWU6JDJ5JDA1JGprbU5CSm1DVTYyV0pKc0tqcWx3RC5BQmpLcUswc2R0SU9lZ29kb3VrQzFpR0owc05ENjhD

K8s ConfigMap configmap.yml

apiVersion: v1  
kind: ConfigMap  
metadata:  
  name: docker-registry  
  namespace: docker-registry  
data:  
  registry-config.yml: |  
    version: 0.1  
    log:  
      fields:  
        service: registry  
    storage:  
      delete:  
        enabled: true  
      cache:  
        blobdescriptor: inmemory  
      filesystem:  
        rootdirectory: /var/lib/registry  
    http:  
      addr: :5000  
      headers:  
        X-Content-Type-Options: [nosniff]  
        Access-Control-Allow-Origin: ['*']  
        Access-Control-Allow-Methods: ['HEAD', 'GET', 'OPTIONS', 'DELETE']  
        Access-Control-Allow-Headers: ['Authorization']  
        Access-Control-Max-Age: [1728000]  
        Access-Control-Allow-Credentials: [true]  
        Access-Control-Expose-Headers: ['Docker-Content-Digest']  
    auth:  
      htpasswd:  
        realm: basic-realm  
        path: /auth/htpasswd  
    health:  
      storagedriver:  
        enabled: true  
        interval: 10s  
        threshold: 3

K8s Deployment deployment.yml

apiVersion: apps/v1  
kind: Deployment  
metadata:  
  name: registry  
  namespace: docker-registry  
spec:  
  replicas: 2  
  selector:  
    matchLabels:  
      app: registry  
  template:  
    metadata:  
      labels:  
        app: registry  
    spec:  
      #restartPolicy: Always  
      volumes:  
        - name: config  
          configMap:  
            name: docker-registry  
            items:  
              - key: registry-config.yml  
                path: config.yml  
        - name: htpasswd  
          secret:  
            secretName: registry-tls  
            items:  
              - key: HTPASSWD  
                path: htpasswd  
        - name: registry-data  
          emptyDir: {}    
      containers:  
        - name: registry  
          image: registry:2.7.1  
          ports:  
            - name: http  
              containerPort: 5000  
              protocol: TCP  
          volumeMounts:  
            - name: config  
              mountPath: /etc/docker/registry  
              readOnly: true  
            - name: htpasswd  
              mountPath: /auth  
              readOnly: true  
            - name: registry-data  
              mountPath: /var/lib/registry

K8s Service service.yml

apiVersion: v1  
kind: Service  
metadata:  
  name: docker-registry  
  namespace: docker-registry  
spec:  
  type: ClusterIP  
  ports:  
    - name: http  
      protocol: TCP  
      port: 5000  
      targetPort: 5000  
  
  selector:  
    app: registry

K8s Ingress ingress.yml

apiVersion: extensions/v1beta1  
kind: Ingress  
metadata:  
  name: docker-registry  
  namespace: docker-registry  
  annotations:  
    #kubernetes.io/ingress.class: nginx  
    #ingress.kubernetes.io/rewrite-target: /  
    #nginx.ingress.kubernetes.io/proxy-body-size: "0"  
    nginx.ingress.kubernetes.io/proxy-body-size: "0"  
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"  
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"  
    #certmanager.k8s.io/issuer: acme-issuer  
spec:  
  #tls:  
  #- hosts:  
  #  - tt.t  
  #  secretName: registry-tls  
  rules:  
  - host: tt.t  
    http:  
      paths:  
      - backend:  
          serviceName: docker-registry  
          servicePort: 5000  
        path: /  
      #- path: /foo  
      #  backend:  
      #    serviceName: service1  
      #    servicePort: 4200

  

yml
---

kubectl create -f xxx.yml
kubectl edit xxx ooo

example：

kubectl create -f secret.yml
kubectl edit secrets –namespace=docker-registry
kubectl edit secrets registry-tls –namespace=docker-registry

kubectl create -f configmap.yml
kubectl edit configmap –namespace=docker-registry
kubectl edit configmap docker-registry –namespace=docker-registry

  

PS
--

containers.volumeMounts mount path must different path. Same path get error: .mountPath: Invalid value: ooxxooxx : must be unique  
  
So configmap

auth.htpasswd.path /etc/docker/registry/htpasswd -> /auth/htpasswd

  

Vaild
-----

  
1\. check Pod realy get config.yml & htpasswd is ok or not? Pod ip use minikube dashboard website more easy.

minikube ssh
curl -v -XGET -u username:password http://172.17.xxx.xxx:5000/v2/_catalog

get 200 OK.  
  
  
2\. Check Service is ok or not ? Service ip use minikube dashboard website more easy.

minikube ssh
curl -v -XGET -u username:password http://10.99.xxx.xxx:5000/v2/_catalog

get 200 OK.  
  
  
3\. 2. Check Ingress is ok or not ? Ingress ip use minikube dashboard website more easy.

minikube ssh
curl -v -XGET -u username:password http://10.0.xxx.xxx:5000/v2/_catalog

get 200 OK.  
  
  
\========  

Ingress
-------

have problems with windows minikube. So now check list：  
  
1、Use it is ingress-nginx  
https://kubernetes.github.io/ingress-nginx/deploy/  
  
2、On ingress-nginx deploy minikube  
  

### Don't USE

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml


### USE

minikube addons enable ingress

Because nginx-control already be create by "minikube addons enable ingress". Check use

kubectl get pods –all-namespaces -l app.kubernetes.io/name=nginx-ingress-controller

PS：Use mandatory.yaml be create ingress-nginx namespace. Use minikube addons be put in kube-system namespace.

https://medium.com/@Oskarr3/setting-up-ingress-on-minikube-6ae825e98f82

Enabling the add-on provisions the following:

．a configMap for the Nginx loadbalancer
．the Nginx ingress controller
．a service that exposes a default Nginx backend pod for handling unmapped requests.


Best Important!!
----------------

Use nginx-ingress that let everything pass nginx-ingress. Thinking Way now is "path" or "domain" to "k8s service"  
Here use "domain" to "k8s service docker-registry"  
EX： tt.t -> nginx-ingress -> service(docker-registry)  
  
Windows use notepad(administrator mode) change C:\\Windows\\System32\\drivers\\etc\\hosts

192.168.99.xxx tt.t

Run curl

curl -v -XGET -u username:password http://tt.t/

get 200. ok  
  

Best Important!!
----------------

Use TLS just remove mark TLS insdie ingress.yml.  
  
Run curl

curl -v -XGET -u username:password https://tt.t/

But maybe get cert can't correct.  
  

Best Important!!
----------------

  
ingress Only domain or path, So http & https. Can't custom port.  
  
https://medium.com/google-cloud/kubernetes-nodeport-vs-loadbalancer-vs-ingress-when-should-i-use-what-922f010849e0  
  

### Maybe

you try to edit Deployments -> nginx-ingress-controller YAML  
  
add custom Port

“ports”: [
…,
{
“hostPort”: 5000,
“containerPort”: 5000,
“protocol”: “TCP”
}
]

Yes, This edit get No error, but ingress

### can't

accept custom port

rules:

http:
port: 5000
paths:
- backend:
  serviceName: docker-registry
  servicePort: 5000
  path: /


### Or

maybe you think Deployments -> nginx-ingress-controller YAML add name for Port. EX:registryhttp 5000  
But ingress

### can't

accept registryport

rules:

registryhttp:
paths:
- backend:
  serviceName: docker-registry
  servicePort: 5000
  path: /

  
Other way is LoadBalancer  

### But only nodePort

apiVersion: v1
kind: Service
metadata:
name: docker-registry-lb
namespace: docker-registry
spec:
type: LoadBalancer
ports:

name: registryhttp
port: 5000
targetPort: 5000
nodePort: 30500
selector:
app: registry

apiVersion: v1
kind: Service
metadata:
name: docker-registry
namespace: docker-registry
spec:
type: NodePort
ports:

name: registryhttp
port: 5000
targetPort: 5000
nodePort: 30500
selector:
app: registry

Port 5000 still

### can't use

### Only Port 30500 can use

  
  

### maybe you think

Use "selector" to focus nginx-ingress-controller

### Answer is can't, because ingress only http & https

  
OK, some demo add port on tcp-services (configmap)  
https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/  
https://yq.aliyun.com/articles/603225

apiVersion: v1
data:
“9000”: docker-registry/docker-registry:5000
kind: ConfigMap
metadata:
creationTimestamp: “2019-01-30T05:24:36Z”
labels:
addonmanager.kubernetes.io/mode: EnsureExists
name: tcp-services
namespace: kube-system
resourceVersion: “256144”
selfLink: /api/v1/namespaces/kube-system/configmaps/tcp-services
uid: 552a9b32-244f-11e9-b70e-080027bbc6ea

  

### Default backend Test

https://kubernetes.github.io/ingress-nginx/user-guide/default-backend/  
  
．/healthz that returns 200  
．/ that returns 404

k8s kubernetes Lesson 2 yaml

PS：Close basic auth just remark configmap.yml auth:

Generate private key & certificate

login ID/PWD

K8s Secret

domain.crt base64

domain.key base64

htpasswd base64

K8s Namespace namespace.json

K8s Secret secret.yml

K8s ConfigMap configmap.yml

K8s Deployment deployment.yml

K8s Service service.yml

K8s Ingress ingress.yml

Sue boy