Eric Guo's blog.cloud-mes.com

in HTML5, JavaScript, Ruby & Rails, Python, and Cloud MES!

Rbenv and CentOS Ruby on Rails Production Environment Install Log

Permalink

Original refer, install on a Aliyun server.

Install software in root account

Update system

yum update
yum install -y htop git zlib zlib-devel gcc-c++ patch readline readline-devel libyaml-devel libffi-devel openssl-devel make bzip2 autoconf automake libtool bison curl sqlite-devel
reboot

Setup a user account

adduser deployer
gpasswd -a deployer wheel
visudo # add deployer ALL=(ALL) NOPASSWD: ALL at end
sudo su - deployer
mkdir .ssh
chmod 700 .ssh

Also disable root login and password via PermitRootLogin in /etc/ssh/sshd_config

Before exis, make sure you can login via ssh deployer@ip_address, other wise, check file permission.

Install rbenv and ruby-build

cd # as a deployer
git clone git://github.com/sstephenson/rbenv.git .rbenv
cd ~/.rbenv && src/configure && make -C src
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile
~/.rbenv/bin/rbenv init
# As an rbenv plugin
mkdir -p "$(rbenv root)"/plugins
git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build

Install Ruby 2.5.1

rbenv install -l
rbenv install 2.5.1
rbenv global 2.5.1
eval "$(rbenv init -)" >> ~/.bash_profile
echo "gem: --no-document" > ~/.gemrc
gem install bundler

Install Javascript Runtime

curl --silent --location https://rpm.nodesource.com/setup_8.x | sudo bash -
sudo yum install nodejs
sudo curl -sL https://dl.yarnpkg.com/rpm/yarn.repo -o /etc/yum.repos.d/yarn.repo
sudo yum install yarn

Install postgresql

sudo yum install postgresql-server postgresql-contrib postgresql-devel
sudo postgresql-setup initdb
sudo systemctl start postgresql
createuser deployer --pwprompt
ALTER ROLE deployer LOGIN
CREATE DATABASE harman_vendor_production WITH ENCODING='UTF8' OWNER=deployer;
/var/lib/pgsql/data/pg_hba.conf
# "local" is for Unix domain socket connections only
local all all peer
psql -d harman_vendor_production

Install nginx

sudo yum install epel-release
sudo yum install nginx

Fix permission for CentOS

sudo mkdir /var/www
cd /var/www
sudo mkdir jbl_product
sudo chown deployer:deployer jbl_product/

or further read nginx permission denied

Deploy Simple Rails App to CentOS 7.4 on Aliyun in 2018

Permalink

It’s 2018, docker quite mature, but since we can buy a server less than 600 RMB in Aliyun including 40Gb storage, 1MB network, 1 core CPU and 2G memory, so I still want to install it in the triditional way.

Install software in root account

Update system

yum update
yum install htop
yum install git
reboot

Install RVM

Resolve can not import from hkp://keys.gnupg.net

In a server which can running below cmd:

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
gpg --export --armor D39DC0E3
gpg --export --armor 39499BDB

In Aliyun server:

gpg --import -

and copy and paste the can run server public key content and press Ctrl+D

Install RVM

\curl -sSL https://get.rvm.io | bash -s stable
source /etc/profile.d/rvm.sh
logout

Update RVM to master

rvm get master
rvm list known # should see ruby 2.5

Install Ruby 2.5

rvm install ruby-2.5
echo "gem: --no-document" >> /etc/gemrc
echo "gem: --no-document" >> ~/.gemrc

Install node.js

yum install nodejs
node --version # v6.12.3

Install yarn

curl --silent --location https://dl.yarnpkg.com/rpm/yarn.repo | sudo tee /etc/yum.repos.d/yarn.repo
yum install yarn

Install nginx

yum install nginx

Normal user

Create new user - harman

adduser --home-dir /data/www/harman harman -g rvm

Copy authorized_keys

mkdir /data/www/harman/.ssh
sudo cp ~/.ssh/authorized_keys /data/www/harman/.ssh/
sudo chown harman:rvm /data/www/harman/.ssh/

Do capistrano deploy

Change nginx

Modify nginx

Add below to http section and comment out default server section.

http {
log_format timed_combined '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" '
'$request_time $upstream_response_time $pipe';
access_log /var/log/nginx/access.log timed_combined;
}

Create /etc/nginx/conf.d/harman.conf

upstream harman_web {
server unix:/data/www/harman/shared/tmp/unicorn.socket fail_timeout=0;
keepalive 3;
}
server {
listen 80 default_server;
# server_name harman.bayekeji.com;
location /nginx_status {
allow 127.0.0.1;
deny all;
stub_status on;
}
root /data/www/harman/current/public;
access_log /data/www/harman/shared/log/harman-access.log timed_combined buffer=1k;
error_log /data/www/harman/shared/log/harman-error.log;
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html break;
}
location ~ (/assets|/uploads|/system|/favicon.ico|/*.txt) {
access_log off;
expires 14d;
gzip_static on;
add_header Cache-Control public;
}
location / {
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_buffering on;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header X-Forwarded-Proto https;
proxy_pass http://harman_web;
gzip on;
}
}

Chart Number and Calculation Logic Explanation for OpenApply

Permalink

I plan to rewrite the OpenApply all the chart, so need list the calculation logic as below first.

Dashboard

The dashboard provides a quick overview of most common key number for the school.

Enquiries

Enquiries provide a month by month overview for inquiry and it’s submitted applicant forms number.

Monthly inquiry students define as the new students created at that month.

Monthly conversion students define as the student who submits the application form that month.

So normally, such gap between inquiries and conversion should not very large, otherwise, it’s indicate maybe your application form is too complex and many students don’t finish it.

Applicants

Applicants will list the students’ number per day, the students is the one who was being marked at the applied status in that day, the mark effective date can be different with actually the marking day.

Applicants date range always starts at Month Day 1.

The applicants number is based on student status history effective on date.

Enrolment

Enrolment chart will display the count number of enrolled student per that month. The enrolled day is students status histories enrolled record effective on date.

Re-Enrolment

Re-enrolment help you manage the re-enrolment student status. there is 3 status which will be recorded in student reenrolment status.

Pending means email already sent, but student no response. Confirmed means student confirm the re-enrolment for your select academic year. Declined means student read the email but decide not continue attending the school at the select academic year.

If the student is eligible to attend the select academic year, but no student reenrolment status record, it will belong to Not sent.

Analytics

Enquiry

Based on filter condition, to analyze the company, nationality by reference source or grade distribution.

The time range always the academic year.

Conversion Funnel

Conversion Funnel focus on the enroll perspective.

The left part is the drop down student status for the select academic year, the Pending number is 100% percent, then all other status is based on that percent.

In demo side, as the enrolled student in 25%, which means only ¼ student can be enrolled amount all inquiry student.

Center part pie chat compare the declined and enrolled students for that year.

Right small pie chat display sub status for select status students, if the school does not configure their own sub-status, it will leave blank.

Enrolment

This part no chart, so intentionally leave blank.

Checklist

The checklist will give an overview of checklist status per academic year and per program.

Pending means those students not complete the checklist item. Partial means those students have already filled the forms, but not submit yet. Complete means those students finish such checklist item.

Click any part of rectangle box will display detail students in such status in the checklist item.

You can filter to analyze students as the checklist completion data source.

Geography

It will analyze the student country, if the student country is blank, will take parents country instead.

Nationality

It will analyze the student’s nationality.

Languages

It will analyze the student’s language.

Pow Used .dev Will Be Redirect to Https by Google Chrome

Permalink

After Chrome 63 released, Google makes the decision to redirect .dev to https direct as they buy the domain TLD.

So if you using pow as well like me, you need to add below line to ~/.profile:

export POW_DOMAINS=localhost

And running below command and restart pow.

cd /etc/resolver/
sudo mv dev localhost
launchctl unload -w ~/Library/LaunchAgents/cx.pow.powd.plist
sudo launchctl unload -w /Library/LaunchDaemons/cx.pow.firewall.plist
sudo pow --install-system
pow --install-local
sudo launchctl load -w /Library/LaunchDaemons/cx.pow.firewall.plist
launchctl load -w ~/Library/LaunchAgents/cx.pow.powd.plist

So you can continue using http://faria.oa.localhost/ to access your local rails projects.

Ruby Conf China 2017 Notes

Permalink

第一场 m-ruby on nginx from strikely

Performance Result

目测删除了对m-ruby的结果,比如OpenResty的测试。

XSS via blog using m-ruby

可以通过nginx层改json的格式,通过mruby语法,性能未知。

Real usage in strikely

Image compress, jwt token verify

Sample setup code

https://github.com/tylerdiaz/ngx-ruby

Limitations

1.9.3 syntax only and can not require files and existing ruby gems.

第二场 Ethereum on Ruby

Rust和Ruby很有渊源,因为它们开头都是Ru。。。

区块链组成:加密算法、P2P网络、共识算法、可证实的数据结构(区块链 Authenticated Data Structure)。

Contract Account是的机器在历史上第一次财务独立,可以代替一切中间人的角色。

EVM Ethereum Virtual Machine 计费力度可以到指令。

交易一般在3秒内传遍世界,EVM跑交易,15秒出一个区块块。

Solidity, Viper(类Python,类型更安全), Bamboo(做形式化证明)

Precompile Contract 预编译的合约用于加速处理,都是加密相关的功能。

第三场 Ruby异步编程奥德赛

全程无尿点,强烈推荐。

第四场 Erlang开发web框架

浪费一个小时。。

第五场 Ruby Web实时通讯方案剖析 侯俊杰

NChan方案。

第六场 Exploring ActiveRecord

第七场 Docker 发布

第八场 Ruby-Packer

Based on SquashFS

第九场 Elixir

第十场 金数据鉴黄

tesseract-ocr 字符识别

第十一场 mobx-ruby

New EC2 Box on Amazon Linux Install Log

Permalink

Install postgresql 9.6

sudo yum update
# https://yum.postgresql.org/repopackages.php#pg96
sudo yum install https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-6-x86_64/pgdg-ami201503-96-9.6-2.noarch.rpm
sudo yum install -y postgresql96 postgresql96-server postgresql96-libs postgresql96-contrib
sudo /etc/init.d/postgresql-9.6 initdb
sudo /etc/init.d/postgresql-9.6 start
sudo chkconfig postgresql-9.6 on
sudo -u postgres -i psql -c 'SELECT version();'

Install nginx

sudo yum install -y nginx
sudo chkconfig nginx on

Install redis

See https://medium.com/@andrewcbass/install-redis-v3-2-on-aws-ec2-instance-93259d40a3ce

Install node.js

curl --silent --location https://rpm.nodesource.com/setup_6.x | bash -
yum install -y gcc-c++ make
yum install -y nodejs

Install Go

wget https://storage.googleapis.com/golang/go1.11.2.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.11.2.linux-amd64.tar.gz
echo "export PATH=$PATH:/usr/local/go/bin" > /etc/profile.d/go-lang.sh

Notes, if you upgrade new version, need rm -rf /user/local/go first.

Install shadowsocks-go

sudo yum install -y git
go get github.com/shadowsocks/shadowsocks-go/cmd/shadowsocks-server
wget -4qO- "http://whatismyip.akamai.com/" # record public IP

Build config.json

{
"server":"127.0.0.1",
"server_port":8388,
"local_port":1080,
"password":"barfoo!",
"method": "aes-128-cfb-auth",
"timeout":600
}

cd ~/go/bin && nohup ./shadowsocks-server &

Install Mosh

sudo yum --enablerepo=epel install -y mosh

Using POW and Byebug Together

Permalink

Some rails application, especially in SaSS category will behave differently based on the different subdomain. In that case, pow is necessary, as it makes simulate subdomain much easier compare with edit based on /etc/hosts.

Setup the pow relative easy:

brew install pow
mkdir -p ~/Library/Application\ Support/Pow/Hosts
ln -s ~/Library/Application\ Support/Pow/Hosts ~/.pow
pow --install-local
launchctl load -w ~/Library/LaunchAgents/cx.pow.powd.plist

Then you can link required rails application via:

cd ~/.pow
ln -s /Users/guochunzhong/git/product_hunt/ product-hunt

And finally, using http://product-hunt.dev to access rails application without using rails s.

Because now there is no terminal to running rails s, so you now have to use tail -f log/development.log to monitor, it’s not a big deal, in fact, more flexible as it enables you Ctrl+C any time and leaves the interesting log in the terminal to read.

The big issue is now you are no access the Byebug as even Byebug is paused, we have no access to it, so must using remote mode of byebug.

Luckily, it’s still not too hard, just following product_hunt commit will do the job.

Explore Rails 5.1 New Feature via an Simple Project

Permalink

Rails 5.1即将发布,虽然同Rails 5相比,这只是一个0.1的小版本,但我看来,Rails 5.1的新的功能却非常重要:在JavaScript世界建构和模块管理工具突飞猛进了3~4年后(grunt, gulp, browserify, npm, webpack, yarn),一向喜欢稳定的Rubyist们(喜欢折腾的都跑去javascript, go和Elixir了。。),在大神DHH和女神Liceth以及其他众神的帮助下,迎来了和node.js国的最终和解:YarnWebpack已经正式成为了主厨推荐

本文和其他介绍Rails 5.1 Whatsnew稍有不同的地方在于,我试图通过一个新的项目探索Rails 5.1的特点。

product hunt是一个功能简单的Rails应用,仅仅实现了产品的维护,但在功能方面,探索了yarn下对新javascript框架的使用,还有集成测试。

具体来说,使用了比较小众的milligram CSS框架,输入自动提示使用了零依赖的awesomplete,使用基于Chrome的集成测试。

Yarn

Yarn同npm相比,优点很多,yml格式的yarn.lock文件显示依赖关系清晰易读,安装速度快,完全不需要再使用asset pipe line对已有的javascript模块进行二次封装,省事省力,省开新gem。环境准备也相当容易,在国内(必须)用淘宝源即可。

brew install yarn
yarn config set registry 'https://registry.npm.taobao.org'

设置完毕后,就可以直接使用yarn添加依赖了。

yarn add milligram

使用CSS前端框架,直接引用即可:

/* app/assets/stylesheets/application.css */
*= require milligram
// app/assets/stylesheets/milligram.sass
@import milligram/src/Color
@import milligram/src/Base
@import milligram/src/Button

使用javascript库多做一步:

// app/assets/config/manifest.js
//= link awesomplete/awesomplete.js
<%# app/views/products/_form.html.erb %>
<%= javascript_include_tag "awesomplete", async: true -%>

Webpack

Webpack是可选的,如果是majestic monolith的Rails应用的话,肯定不会用,但是现代的前后端分离风潮下,至少Rails 5.1让这个前后端分离变得无比容易,比之前的react_on_rails方案容易好多。

jQuery的依赖去除

由于Javascript一众MVC框架的崛起,Rails 5.1在前端方案上给予了开发者更多的选择权,我试了在没有jquery的情况下,使用0 payloading的vanilla-js开发,虽然时不时还需要查一下对应jQuery的用法,但这样做还是值得的,微信小程序的卖点就是即用即走,但如果能将网页做到轻量,何尝不是即用即走?况且网页发布还不用走审核流程。

在Rails下使用vanilla js其实也不是什么都没得用,rails-ujs始终是存在的,所以还是可以直接使用Rails.ajax方法来发起远程调用:

window.addEventListener('input', function (e) {
if (e.target.id == "product_name") {
Rails.ajax({
type:'GET',
url: e.target.getAttribute('data-url'),
dataType: 'script'
});
}
}, false);

System Test

Rails 5.1对集成测试也有了官方方案,现在出厂即可跑集成测试。集成测试在测试自动提示这样的特性的时候还是很方便的,官方出厂的默认配置基于selenium-webdriver驱动的Chrome,使用之前需要安装驱动:

brew install chromedriver
require 'application_system_test_case'
class NewProductTest < ApplicationSystemTestCase
test 'create a new product' do
visit '/products/new'
fill_in 'product_name', with: 'L'
page.has_selector?('ul > li > mark')
fill_in 'product_name', with: 'Le Wagon'
page.has_no_selector?('ul > li > mark')
fill_in 'product_tagline', with: 'Change your life: Learn to code'
click_button 'Create Product'
# Should be redirected to Home with new product
assert_equal product_path(Product.last), page.current_path
assert page.has_content?('Change your life: Learn to code')
end
end

这里第一次在product_name填入‘L',应该有自动完成的提示出现,完全输入后,则没有,这些都可以通过Capybara的浏览器DSL做检测,从而在集成测试中一并测试好,相比之前的单个controller测试,效率提高不少。

其他特性。

其他的特性:Encrypted secrets、Parameterized mailers、Direct & resolved routes等,参见Rails 5.1 Release notes,演示项目中没有包括这些功能,这里就略过了。

本文不足

测试方案还想使用phantomjs和poltergeist,但是始终没有成功,有兴趣的同学可以尝试并提Pull Request。