Eric Guo's blog.cloud-mes.com

Hoping writing JS, Ruby & Rails and Go article, but fallback to DevOps note

Eric-Guo's Macbook Shortcut Cheetsheet

Permalink

CMD keys

Shortcut Description
cmd+ctl+Q Lock screen

FN keys

Shortcut Description
fn+A Navigate Dock
fn+C Control Center
fn+E Emoji & Symbols
fn+F Enter/Exit Full Screen
fn+H Desktop
fn+M Navigate Main Menu
fn+N Notifications
fn+Q Quick Note

original

shortcut in Finder

Shortcut Description
cmd+opt+P Show/Hide Path bar

Quick Clean Server Disk Usage Command in Safe Way

Permalink

SystemD journal clean

journalctl --disk-usage
sudo journalctl --vacuum-size=2000M # Delete log files until the disk space taken falls below the specified size:

Or see further command

Apt cache clean

sudo apt-get clean

Yum cache clean

sudo yum clean all

Yarn cache clean

yarn cache clean --force

Docker cache clean

docker system df
docker system prune -a

original from stackoverflow

My Monthly Subscription Review List

Permalink

No one like subscription, but we must pay something we really need every month and here is my list in RMB:

  1. (21) iCloud 200GB
  2. (15) Apple Music
  3. (23) Dragonruby Pro (annual 42 USD)
  4. (33) bandwagon host (month 33 USD)
  5. (6) Adblock Pro (annual 70 RMB)
  6. (6) MarginNote 3 OCR (annual 68 RMB)
  7. (28) Rubymine (annual 53 USD)
  8. (27) Medium Member (annual 50 USD)
  9. (6) blog domain (annual 10 USD)
  10. (1.5) 香哈菜谱 (annual 18 RMB)
  11. (10) AWS hosting
  12. (16.8) Meituan biking
  13. (50) Slack (month 8 USD)
  14. (38) Google Workspace Business Starter (month 6 USD)

So totally 282 RMB per month have to pay.

Setup Advanced Search in Gitlab 14.7 Log

Permalink

Installation Elastic Search using Debian/Ubuntu repository

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo apt-get install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list
sudo apt-get update && sudo apt-get install elasticsearch
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch.service
sudo systemctl start elasticsearch.service
curl localhost:9200/

Enable correct IP and other securty settings:

/etc/elasticsearch/elasticsearch.yml
cluster.name: gitlab-elasticsearch
network.host: 192.168.100.20
http.port: 9200
discovery.seed_hosts: ["192.168.100.20"]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

Initial password and record it down.

/usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto

Install analysis-smartcn

cd /usr/share/elasticsearch/
sudo bin/elasticsearch-plugin install analysis-smartcn
sudo systemctl restart elasticsearch

Setup a Gitlab Runner Server on Ubuntu 20.04

Permalink

According to the official documentation, I having a free lighthouse server from tencent and install Ubuntu 20.04 to running gitlab runner.

Install docker.

sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt-get install ca-certificates curl gnupg lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
docker run hello-world

Install gitlab runner

sudo su -
docker run --rm -it -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner --help
docker run --rm -it -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register
docker run --rm -it -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner verify
docker run -d --name gitlab-runner --restart always -v /srv/gitlab-runner/config:/etc/gitlab-runner -v /var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:latest
docker logs gitlab-runner
docker restart gitlab-runner

Upgrade gitlab runner

sudo su -
docker stop gitlab-runner && docker rm gitlab-runner
docker pull gitlab/gitlab-runner
docker run -d --name gitlab-runner --restart always -v /srv/gitlab-runner/config:/etc/gitlab-runner -v /var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:latest
docker logs gitlab-runner
docker restart gitlab-runner

Fix the docker address pools conflict with internal network IP

docker network inspect bridge
vi /etc/docker/daemon.json
systemctl restart docker.service
docker network inspect bridge
{
"registry-mirrors": ["https://mirror.ccs.tencentyun.com", "http://hub-mirror.c.163.com"],
"default-address-pools":
[
{"base":"10.200.100.0/16","size":24}
]
}

Clean docker cache

wget https://gitlab.com/gitlab-org/gitlab-runner/-/raw/main/packaging/root/usr/share/gitlab-runner/clear-docker-cache
chmod +x clear-docker-cache
./clear-docker-cache

Further info

Install Rails Stack for Ubuntu 20.04

Permalink

Install nginx

sudo su -
cd /etc/apt/sources.list.d/
vi nginx.list
apt update

And paste the source:

nginx.list
deb https://nginx.org/packages/ubuntu/ focal nginx
deb-src https://nginx.org/packages/ubuntu/ focal nginx

And public keys:

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys ABF5BD827BD9BF62
apt update
apt-get install nginx
systemctl start nginx
systemctl enable nginx

It’s also possible to enable TLS 1.1/1.2 due to Ubuntu 20.04 not support by default.

/etc/nginx/nginx.conf
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
# seclevel for TLS 1.0 and 1.1
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:@SECLEVEL=1";

Install node.js 16 and yarn 1.x

Using nodesource distribution

curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt-get install gcc g++ make git
sudo apt-get install -y nodejs
## To install the Yarn package manager
curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor | sudo tee /usr/share/keyrings/yarnkey.gpg >/dev/null
echo "deb [signed-by=/usr/share/keyrings/yarnkey.gpg] https://dl.yarnpkg.com/debian stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt-get update && sudo apt-get install yarn

Install rbenv

sudo su - # as root
apt update
apt install build-essential libgdbm-dev libncurses-dev libreadline-dev libssl-dev libyaml-dev zlib1g-dev
logout # as ubuntu
# git clone https://github.com/rbenv/rbenv.git ~/.rbenv
git clone https://gitee.com/Eric-Guo/rbenv.git ~/.rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
source ~/.bashrc
type rbenv
# git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build
git clone https://gitee.com/Eric-Guo/ruby-build.git "$(rbenv root)"/plugins/ruby-build
# git clone https://github.com/andorchen/rbenv-china-mirror.git "$(rbenv root)"/plugins/rbenv-china-mirror
git clone https://gitee.com/Eric-Guo/rbenv-china-mirror.git "$(rbenv root)"/plugins/rbenv-china-mirror
rbenv install -l

Install ruby 3.0.3

rbenv install 3.0.3
rbenv global 3.0.3
echo "gem: --no-document" > ~/.gemrc
rbenv shell 3.0.3
gem update --system
gem install bundler
gem install bundler --default
gem install bundler

Install postgresql

apt-get install postgresql
apt-get install postgresql-server-dev-all
sudo su - postgres
createuser ubuntu
psql
ALTER ROLE ubuntu LOGIN;
CREATE DATABASE wefocusin_production WITH ENCODING='UTF8' OWNER=ubuntu;
logout
psql -d wefocusin_production -f wefocusin_production.sql

Install certbot

sudo apt install snapd
sudo snap install core
sudo snap refresh core
sudo snap install --classic certbot
sudo ln -s /snap/bin/certbot /usr/bin/certbot
sudo certbot --nginx

Using Podman-docker in Tlinux - TencentOS Version 3.1

Permalink

Notice gitlab still need using real docker instead of podman-docker and below is only record as a testing purpose and failed, detail see gitlab issue 27119.

sudo yum install podman-docker
sudo mv /etc/containers/registries.conf /etc/containers/registries.conf.orig
docker run --rm -t -i gitlab/gitlab-runner --help
sudo mkdir -p /srv/gitlab-runner/config
docker run --rm -it -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register
docker run --rm -it -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner verify
/etc/containers/registries.conf
unqualified-search-registries = ["docker.io"]
[[registry]]
prefix = "docker.io"
insecure = false
blocked = false
location = "docker.io"
[[registry.mirror]]
location = "hub-mirror.c.163.com"
[[registry.mirror]]
location = "registry.docker-cn.com"

Install Gitlab 14.6 on Ubuntu 20.04 Log

Permalink

Installation gitlab from source is not easy, but it’s worth as it enable me do some edit latter, so here is my installation log.

Create git user and make git user sudo

adduser deployer
sudo su - deployer
mkdir .ssh
chmod 700 .ssh
vi .ssh/authorized_keys # and paste your public key
chmod 600 .ssh/authorized_keys
sudo su -
cd /etc/sudoers.d/
echo "deployer ALL=(ALL) NOPASSWD:ALL" > 20-deployer-user

Install build dependencies

sudo apt remove libcurl4 # see https://askubuntu.com/a/1134120/111945
sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libre2-dev \
libreadline-dev libncurses5-dev libffi-dev curl openssh-server libxml2-dev libxslt-dev \
libcurl4-openssl-dev libicu-dev logrotate rsync python-docutils pkg-config cmake runit-systemd

Install Ruby 2.7.6

sudo apt install git autoconf bison
mkdir /tmp/ruby && cd /tmp/ruby
curl --remote-name --progress-bar "https://cache.ruby-china.com/pub/ruby/2.7/ruby-2.7.6.tar.gz"
echo 'e7203b0cc09442ed2c08936d483f8ac140ec1c72e37bb5c401646b7866cb5d10 ruby-2.7.6.tar.gz' | sha256sum -c - && tar xzf ruby-2.7.6.tar.gz
cd ruby-2.7.6
./configure --disable-install-rdoc --enable-shared
make
sudo make install
gem update --system
gem install bundler
gem install --default bundler

Install Go 1.17

wget https://go.dev/dl/go1.17.13.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.17.13.linux-amd64.tar.gz
sudo vi /etc/environment # add :/usr/local/go/bin in PATH
go version

Install Git in gitlab versions

sudo apt-get install -y libcurl4-openssl-dev libexpat1-dev gettext libz-dev libssl-dev libpcre2-dev build-essential git-core
git clone https://e.coding.net/sew/gitlab/gitaly.git -b 14-6-stable /tmp/gitaly
cd /tmp/gitaly
sudo make git GIT_PREFIX=/usr/local
sudo apt remove -y git-core git
sudo apt autoremove

Install Node.js 14 and other depends

sudo apt-get install -y graphicsmagick
sudo apt-get install -y postfix
sudo apt-get install -y libimage-exiftool-perl
curl --location "https://deb.nodesource.com/setup_14.x" | sudo bash -
sudo apt-get install -y gcc g++ make
sudo apt-get install -y nodejs
curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor | sudo tee /usr/share/keyrings/yarnkey.gpg >/dev/null
echo "deb [signed-by=/usr/share/keyrings/yarnkey.gpg] https://dl.yarnpkg.com/debian stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt-get update && sudo apt-get install yarn

System users

sudo adduser --disabled-login --gecos 'GitLab' git

Database

sudo apt install -y postgresql postgresql-client libpq-dev postgresql-contrib

And following the official documentation section.

Redis

sudo apt-get install redis-server

And following the official documentation section.

Clone Gitlab and do some change

Largely following official document.

sudo -u git -H git clone https://e.coding.net/sew/gitlab/gitlab.git -b thape_deploy gitlab
  • need change database.yml to using local user
  • need filling the config/secrets.yml with proper secrets by running rails secret
  • need setting the proxy for go
# setting go proxy
export GO111MODULE=on
export GOPROXY=https://goproxy.cn
sudo chown deployer:deployer -R .bundle/
bundle install
sudo chown git:git -R .bundle/

Install GitLab Shell

cd /usr/bin
sudo ln -s /usr/local/bin/git git
sudo -u git -H bundle exec rake gitlab:shell:install RAILS_ENV=production
cd /home/git/gitlab-shell/
sudo -u git -H make
cd /home/git/gitlab/
sudo -u git -H bundle exec rake gitlab:shell:install RAILS_ENV=production

Install GitLab Workhorse

sudo -u git -H bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workhorse,https://e.coding.net/sew/gitlab/gitlab-workhorse]" RAILS_ENV=production
cd /home/git/gitlab-workhorse
sudo -u git -H git checkout -b 8-65-stable
cd /home/git/gitlab
sudo -u git -H bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workhorse,https://e.coding.net/sew/gitlab/gitlab-workhorse]" RAILS_ENV=production

Install GitLab-Elasticsearch-indexer on Enterprise Edition

sudo su - git
cd /home/git
git clone https://e.coding.net/sew/gitlab/gitlab-elasticsearch-indexer.git
logout
cd /usr/bin
sudo ln -s /usr/local/go/bin/go go
cd /home/git/gitlab
sudo -u git -H bundle exec rake "gitlab:indexer:install[/home/git/gitlab-elasticsearch-indexer,https://e.coding.net/sew/gitlab/gitlab-elasticsearch-indexer.git]" RAILS_ENV=production

Install GitLab Pages

cd /home/git
sudo -u git -H git clone https://e.coding.net/sew/gitlab/gitlab-pages.git
cd gitlab-pages
sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_PAGES_VERSION)
sudo -u git -H make

Install Gitaly

Beside following documents, also need to do a bundler install in ruby folder and configure-authentication.

cd /home/git/gitlab
sudo -u git -H bundle exec rake "gitlab:gitaly:install[/home/git/gitaly,/home/git/repositories,https://e.coding.net/sew/gitlab/gitaly.git]" RAILS_ENV=production

Some configure after install

I only record I do:

  1. trusted-proxies

Backup

cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production
# find backup at ./gitlab/tmp/backups

Using Patch-package to Tuning CSS in 3rd Party Packages

Permalink

Add patch-package

yarn add patch-package postinstall-postinstall

Apply the change

Open sublime-text and modify the source code

Generate the patch

yarn patch-package viewerjs

Auto patch

"private": true,
+ "scripts": {
+ "postinstall": "patch-package"
+ },
"dependencies": {