Eric Guo's blog.cloud-mes.com

Hoping writing JS, Ruby & Rails and Go article, but fallback to DevOps note

Prepare the 16 kHz Monaural MP3 for Wechat API Speach to Text Interface

• Permalink 

ffmpeg -i test_voice.amr -acodec mp3 -ac 1 -ar 16000 test_voice.mp3

微信AI开放接口, require that interface.

How to install in CentOS 7 from source code

yum install nasm

cd /usr/local/src

wget http://downloads.sourceforge.net/lame/lame-3.100.tar.gz

tar -zxvf lame-3.100.tar.gz

cd lame-3.100

./configure --prefix=/usr/local

make && make install

ln -s /usr/local/lib/libmp3lame.so.0.0.0 /usr/lib64/libmp3lame.so.0

wget http://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2

tar -jxvf ffmpeg-snapshot.tar.bz2

cd ffmpeg

./configure --prefix=/usr/local --enable-libmp3lame

make && make install

ffmpeg

Original post

How to Resolve Rails Test:system Long Time Pending Using Chromedriver

• Permalink

Running rails test:system in my local dev always pending and timeout problem haunted me for a long time, finally, I know why.

Due to some change introduced in Google, every time a new chromedriver installed using brew, the first time running always need to download a file from storage.l.googleusercontent.com. but due to there is GFW in China, it always pending.

So resolved is quite simple, make sure you running ruby.exe(or binary of ruby) when first time can access google site and that's it.

You need to make sure only once per every new version, that's it!

Allow New User to Login From Remote in Percona Server

• Permalink

Allow remote connect

vi /etc/my.cnf # bind-address = 0.0.0.0

systemctl restart mysqld

netstat -lnp | grep mysql # confirm 3306 is listened

firewall-cmd --add-service=mysql --permanent

firewall-cmd --reload

Add user with dedicated table space access

mysql -u root -p # input password if required

CREATE DATABASE cybros_bi character set UTF8mb4 collate utf8mb4_bin;

CREATE USER 'cybros_bi'@'%' IDENTIFIED BY 'cybros_bi_password';

GRANT ALL ON cybros_bi.* TO 'cybros_bi'@'%';

FLUSH PRIVILEGES;

Add new user but only read only table access.

mysql -u root -p # input password if required

CREATE USER 'cad_reader'@'%' IDENTIFIED BY 'cad_reader_password';

GRANT SELECT ON cybros_prod.cad_operations TO 'cad_reader'@'%';

GRANT SELECT, INSERT, UPDATE, DELETE ON cybros_prod.split_cost_items TO 'cad_reader'@'%';

FLUSH PRIVILEGES;

Show user access

show grants for 'sync_user_read_only'@'%';

Check all users

SELECT user, host FROM mysql.user;

Install Pentaho Data Integration on MacOS 10.14.5 Mojave

• Permalink

It's a little hard to find out the root cause why out of box Pentaho Data Integration not working, any way, it's resolved.

First need java 8 to install, because Oracle refuse to provide, need OpenJDK instead.

brew tap adoptopenjdk/openjdk

brew cask install adoptopenjdk8

brew install jenv # If also want to using other java version

cd "/Applications/Data Integration.app/Contents/MacOS"

cat JavaApplicationStub

Also need change JavaApplicationStub file as below. (Maybe Pentaho developer didin't having a MBP...)

#!/bin/sh

 
# PROG_DIR=$(cd "$(dirname "$0")"; pwd)

 
# PROG_DIR is in .app/Contents/MacOS

 
# BASE_DIR="$PROG_DIR"/../../../

BASE_DIR="/usr/local/Caskroom/data-integration/8.2.0.0-342/data-integration"

cd "$BASE_DIR"

echo $BASE_DIR

. "spoon.command" "$BASE_DIR"

If you want to continue using Java 12 in system wide, install brew install jenv and running jenv local 1.8 at data-integration folder.

You could also setting below environment variable:

spoon.sh
# . "$BASEDIR/set-pentaho-env.sh"

# setPentahoEnv

# Comment out above 2 line and add below 2 line

_PENTAHO_JAVA_HOME="/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/"

_PENTAHO_JAVA="/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/bin/java"

Using Aliyun OSS as CDN for the Rails Application

• Permalink

用Rails做企业官网虽然不是现在的流行方式，但是考虑到Rails的灵活性和后端开发的方便性，在某些情况下，还是相比现在最火的JAM gatsby.js更实际。

企业官网一般图片等静态资源非常多，但是用Rails的话，由于很低的访问量，去购买一台高带宽的服务器又很不划算，所幸的是，阿里云的OSS提供了回源，通过适当的配置，就可以将那些大图片，大字体移到按流量付费的OSS上，获得极大的速度提升，基本原理如下：

                      [User]

                         |

<https://thape-assets.oss-cn-shanghai.aliyuncs.com/assets/application-digest.js>

                         |

               ---------------------------------------

               |                                     |

            <cache>                             <no cache>

               |                                     |

             [200]                <https://www.thape.com.cn/assets/application-digest.js>

                                                     |

                                           [Nginx location /assets]

                                                     |

                                                   [200] --> [CDN Cache]

配置方法也非常简单，新建一个OSS Bucket，例如上图中的名字thape-assets

最后在Rails中的 config/environments/production.rb 中，启用新的OSS地址即可。

  # Enable serving of images, stylesheets, and JavaScripts from an asset server.

  config.action_controller.asset_host = 'https://thape-assets.oss-cn-shanghai.aliyuncs.com'

CentOS Rails Server With Oracle on VMware Installation Log

• Permalink

First CentOS server installed in VMware as thape SSO server.

Install software in root account

Update system

Run as root:

yum update

yum install -y git git-lfs zlib zlib-devel gcc-c++ patch readline readline-devel libyaml-devel libffi-devel openssl-devel make bzip2 autoconf automake libtool bison curl sqlite-devel

wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

rpm -ivh epel-release-latest-7.noarch.rpm

yum --enablerepo=epel install htop

reboot

As VMware admin required, not disable the PermitRootLogin in /etc/ssh/sshd_config

Setup a user account

adduser deployer

gpasswd -a deployer wheel

visudo # add deployer ALL=(ALL) NOPASSWD: ALL at end

sudo su - deployer

mkdir .ssh

chmod 700 .ssh

vi .ssh/authorized_keys

chmod 600 .ssh/authorized_keys

Also disable root login and password via PermitRootLogin in /etc/ssh/sshd_config

Before exis, make sure you can login via ssh deployer@ip_address, other wise, check file permission.

Install rbenv and ruby-build

cd # as a deployer

git clone https://github.com/sstephenson/rbenv.git .rbenv

cd ~/.rbenv && src/configure && make -C src

echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile

~/.rbenv/bin/rbenv init

# As an rbenv plugin

mkdir -p "$(rbenv root)"/plugins

git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build

git clone https://github.com/andorchen/rbenv-china-mirror.git "$(rbenv root)"/plugins/rbenv-china-mirror

Install Ruby 2.6.5

rbenv install -l

rbenv install 2.6.5

rbenv global 2.6.5

echo 'eval "$(rbenv init -)"' >> ~/.bash_profile

echo "gem: --no-document" > ~/.gemrc

gem install bundler

gem install bundler -v 1.17.3

Install Javascript Runtime

Run as root:

curl -sL https://rpm.nodesource.com/setup_10.x | bash -

sudo yum install nodejs

curl -sL https://dl.yarnpkg.com/rpm/yarn.repo | sudo tee /etc/yum.repos.d/yarn.repo

sudo yum install yarn

# if you behind GFW

# But notice sometime some packages not sync so restore back via

# npm config set registry https://registry.npmjs.com/ --global

# npm config delete disturl --global

npm config set registry https://registry.npmmirror.com/ --global

npm config set disturl https://npmmirror.com/dist --global

# Sometime taobao is out of sync, so still need official registry.

# yarn config delete registry --global

yarn config set registry https://registry.npmmirror.com/ --global

yarn config set disturl https://npmmirror.com/dist --global

Install nginx

sudo yum install epel-release

sudo yum install nginx

chkconfig nginx on

sudo firewall-cmd --add-service=http --permanent

sudo firewall-cmd --add-service=https --permanent

sudo firewall-cmd --reload

Fix permission for CentOS

sudo mkdir /var/www

cd /var/www

sudo mkdir cybros

sudo chown deployer:deployer cybros/

and disable selinux(https://linuxize.com/post/how-to-disable-selinux-on-centos-7/),

or further read nginx permission denied

Install Oracle Instant Client

Download Version 12.2.0.1.0 and following ruby-oci8 document

sudo rpm -i ./oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm

sudo rpm -i ./oracle-instantclient12.2-devel-12.2.0.1.0-1.x86_64.rpm

sudo rpm -i ./oracle-instantclient12.2-sqlplus-12.2.0.1.0-1.x86_64.rpm

cd /usr/local/bin

sudo ln -s /usr/bin/sqlplus64 sqlplus

export LD_LIBRARY_PATH=/usr/lib/oracle/12.2/client64/lib

gem install ruby-oci8
Append to ~/.bashrc
export LD_LIBRARY_PATH=/usr/lib/oracle/12.2/client64/lib

export NLS_LANG=AMERICAN_AMERICA.AL32UTF8

Install FreeTDS to connect to SQL Server

sudo su -

wget ftp://ftp.freetds.org/pub/freetds/stable/freetds-1.00.111.tar.gz

tar -xzf freetds-1.00.111.tar.gz

cd freetds-1.00.111

./configure --prefix=/usr/local --with-tdsver=7.3

make

sudo make install

logout # as deployer

gem install tiny_tds

Install MySQL

Install the percona server via yum.

After install, do the secure installation for root.

mysql -u root -p

CREATE DATABASE pp_prod character set UTF8mb4 collate utf8mb4_0900_ai_ci;

CREATE USER 'pp_vendor'@'localhost' IDENTIFIED BY 'new_password';

GRANT ALL ON pp_prod.* TO 'pp_vendor'@'localhost';

FLUSH PRIVILEGES;

Install Ruby on Rails Production Based on Amazon Linux 2 AMI (HVM)

• Permalink

Based my last rbenv and CentOS Ruby on Rails production environment install log, but this time on AWS!

But will only record the difference as below:

Run sudo amazon-linux-extras install nginx1.12 instead of sudo yum install nginx to install nginx.

Seems AMI provide their own version of below package.

  0  ansible2                 available    [ =2.4.2  =2.4.6 ]

  2  httpd_modules            available    [ =1.0 ]

  3  memcached1.5             available    [ =1.5.1 ]

  4  nginx1.12=latest         enabled      [ =1.12.2 ]

  5  postgresql9.6            available    [ =9.6.6  =9.6.8 ]

  6  postgresql10             available    [ =10 ]

  8  redis4.0                 available    [ =4.0.5  =4.0.10 ]

  9  R3.4                     available    [ =3.4.3 ]

 10  rust1                    available    \

        [ =1.22.1  =1.26.0  =1.26.1  =1.27.2  =1.31.0 ]

 11  vim                      available    [ =8.0 ]

 13  ruby2.4                  available    [ =2.4.2  =2.4.4 ]

 15  php7.2                   available    \

        [ =7.2.0  =7.2.4  =7.2.5  =7.2.8  =7.2.11  =7.2.13  =7.2.14 ]

 16  php7.1                   available    [ =7.1.22  =7.1.25 ]

 17  lamp-mariadb10.2-php7.2  available    \

        [ =10.2.10_7.2.0  =10.2.10_7.2.4  =10.2.10_7.2.5

          =10.2.10_7.2.8  =10.2.10_7.2.11  =10.2.10_7.2.13

          =10.2.10_7.2.14 ]

 18  libreoffice              available    [ =5.0.6.2_15  =5.3.6.1 ]

 19  gimp                     available    [ =2.8.22 ]

 20  docker=latest            enabled      \

        [ =17.12.1  =18.03.1  =18.06.1 ]

 21  mate-desktop1.x          available    [ =1.19.0  =1.20.0 ]

 22  GraphicsMagick1.3        available    [ =1.3.29 ]

 23  tomcat8.5                available    \

        [ =8.5.31  =8.5.32  =8.5.38 ]

 24  epel                     available    [ =7.11 ]

 25  testing                  available    [ =1.0 ]

 26  ecs                      available    [ =stable ]

 27  corretto8                available    [ =1.8.0_192  =1.8.0_202 ]

 28  firecracker              available    [ =0.11 ]

 29  golang1.11               available    [ =1.11.3 ]

 30  squid4                   available    [ =4 ]

 31  php7.3                   available    [ =7.3.2 ]

 32  lustre2.10               available    [ =2.10.5 ]

OpenID Connect Learning Notes

• Permalink

最近在做面向大中型企业的 oauth2id 方案，这个领域有很多术语，比如OAuth2, OpenID Connect，同时也有很多相同概念的术语，例如SSO（Single Sign On），CAS（Central Authentication Service）

中文的资料相当少，入门性的有阮一峰的理解OAuth 2.0，英文资料较多：

这篇介绍了ASP.net端的

https://andrewlock.net/an-introduction-to-openid-connect-in-asp-net-core/

openid的FAQ也值得一读：

https://openid.net/connect/faq/