Eric Guo's blog.cloud-mes.com

in HTML5, JavaScript, Ruby & Rails, Python, and Cloud MES!

Design Resource & Tips

Permalink

A list of design resource from lewagon

UI do’s and don'ts

https://goodui.org/

Pictures

Colors

Background gradient

UI gradients

Fonts - Google fonts

Make your shopping on Google fonts.

For main typo

  • Open-Sans

For headers

  • Raleway
  • Montserrat
  • Varela Round
  • Museo

Fonts - Hack on Github

Look for .ttf/.eot/.woff/etc.. files on Github.

  • Avenir
  • Proxima Nova
  • Gotham Rounded
  • Brandon

Icons

HTML/CSS snippets

Inspiration

在Macbook Pro 2016中原生安装tensorflow 1.0

Permalink

前几天的TensorFlow开发者峰会宣布的TensorFlow 版本达到1.0,再加上过去一年人工智能,机器学习非常非常热,虽然作为一名Ruby程序员,但形势所迫,又不得不捡起了Python,准备稍稍了解一下。

TensorFlow的安装官网写的很清晰,我选择的是virtualenv的安装,安装的python版本是brew的版本。

brew install python
sudo easy_install pip
sudo pip install --upgrade virtualenv
virtualenv --system-site-packages ~/tensorflow
source ~/tensorflow/bin/activate
pip install --upgrade tensorflow

安装完成后每次运行需要先运行source ~/tensorflow/bin/activate,嫌麻烦可以把这句加到.bash_profile中。

echo "source ~/tensorflow/bin/activate" >> ~/.bash_profile

安装完成后可以跑一下tensorflow版的“Hello Word”验证一下安装是否正确,需要先进入python的交互环境:

python

然后运行:

import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))

问题来了,一堆CPU指令没有优化,Macbook Pro 2016已经没有核弹卡了,只能跑在CPU版上了,如果CPU再不优化,感觉已经很难愉快的玩下去了,所以只能选择从源码安装了。

源码安装就很烦了,下载源吗就非常费网络,还必须FQ,编译也需要至少25分钟时间,不过亲爱的读者,我已经帮你们做好了,现在可以使用我的安装包直接升级到原生CPU优化版:

pip install --upgrade http://baye-deploy.oss-cn-shanghai.aliyuncs.com/tensorflow-1.0.0-cp27-cp27m-macosx_10_12_x86_64.whl

巴爷供销社,一个生产级别的微信小应用商城开源实现

Permalink

今天是微信小应用发布的日子,很高兴巴爷科技(上海)有限公司能作为第一批微信小应用的Rails厂商,能为Rails社区提供一个开源的参考实现。(包括后端)

微信小应用的介绍文章已经很多了,快速的开发体验,iOS/Android不必分别开发俩次都是很大的优点。但同时,腾讯微信也不是上次的公众号发布时的微信了,7亿的用户,近乎100%的中国市场,所以腾讯变的非常非常小心谨慎,所以也不要指望能像原来的公众号,能从腾讯分到近乎免费的流量红利。

微信小程序限制非常之多:微信小程序二维码只能通过扫一扫进入,不能通过图片扫描;微信小程序没有应用商店,只能通过发现,小程序的的搜索功能寻找;更令人发指的是,非合作伙伴的话,还必须搜索全名才行,也就是说,只有腾讯投资的京东,才能搜索“京东”找到“京东购物”,像巴爷科技,搜索“巴爷”是无法得到任何结果的,必须搜索“巴爷供销社”。

微信小应用在开发方面,微信基本保持了和服务号近似的接口,例如微信支付的对接。但也有不同的地方,例如敏感数据的加解密等。另外由于我们开始开发等时间较早,持续到现在,很多API不断发生改变,这也确实给我们的开发造成了一些困扰。

微信小应用递交审核方面,由于我们不是内测的200个名单中的公司,而是作为第二批公测用户,递交审核的周期和苹果差不多,一次审核周期在1~2天,甚至周末也有人审核。但不好的地方在于,拒的话,只会告诉你一个问题,而且一旦发现应用有bug,有和应用说明中没写的功能,超出微信小应用所选分类的功能,都会被拒。所以如果你的应用如果想快速上线,基本很难。审核期间我们在是否需要ICP证方面被拒了3次,中国的政策很奇怪,理论上只要你有营利性的网上经营行为就必须要ICP证,否则就属于非法经营。但是如果你到电信管理局去申请这个ICP证,办事人员却会以你的公司业务量太小,拒绝你的申请(🐷要养肥了杀?),但好在腾讯在最后一刻,终于在审核上线时,不需要微信小应用递交ICP证了,所以,现在能访问到“巴爷供销社”,还是要感谢腾讯在政府关系方面做出的努力。

微信小程序的github开源地址:https://github.com/bayetech/wechat_mall_applet

后端基于active_model_serializers的参考实现地址:https://github.com/bayetech/wechat_mall_applet_backend

A Rubyist Web Developer Macbook Pro With Touch Bar Installation Notes

Permalink

Just got the Macbook Pro 2016 with touch bar 15’ edition, setting it up takes my 1 full day. If the China network condition is good, I believe it will takes much less time.

I decide the Macbook Pro from scratch instead of using migration assist because I want to keep the system clean. I will install as much application as possible from App Store, then the Homebrew, if both software repositry can not find it, install it manually.

Apple store installation is the easiest one, once you login and you will got all the application you have purchased.

brew are also relative easier, I even using brew to install Chrome and Firefox, via brew cask install google-chrome.

I would like to using percona instead of mysql because in China, Aliyuj RDS is largely based on percona, so I perfer using same software in dev env.

But you can not simply install percona directly.

Install mysql, then percona to avoid pitfall

brew install mysql
brew services start mysql
brew services stop mysql
brew remove mysql
brew install percona-server
brew services start percona-server

twitter cli need to rename tt instead of t

gem install t
mv /usr/local/bin/t /usr/local/bin/tt

Nokogiri using system library

brew install libxml2
# If installing directly
gem install nokogiri -- --use-system-libraries \
--with-xml2-include=$(brew --prefix libxml2)/include/libxml2
# If using Bundle
bundle config build.nokogiri --use-system-libraries \
--with-xml2-include=$(brew --prefix libxml2)/include/libxml2
bundle install

proxychains using Shadowsocks

To add Shadowsocks setting for proxychains.

vi /usr/local/Cellar/proxychains-ng/4.11/etc/proxychains.conf

It will finally takes 76Gb after install all my needs, about 33% used, so I think buy 256Gb SSD edition of rMBP should be enought for my next 4 years usage.

Deploy Ruby-china as a New Forum Software to Aliyun

Permalink

This wiki based on the Ubuntu 14.04.5 LTS Server.

Prerequisites

apt-get update && apt-get upgrade
apt-get dist-upgrade
apt-get install software-properties-common htop curl git

memcached

apt-get install memcached

Redis

add-apt-repository ppa:chris-lea/redis-server
apt-get update && apt-get install redis-server

PostgreSQL

echo "deb http://apt.postgresql.org/pub/repos/apt/ trusty-pgdg main" > /etc/apt/sources.list.d/pgdg.list
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
apt-get update && apt-get install postgresql
apt-get install libpq-dev

imagemagick

apt-get install imagemagick

node.js

curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
apt-get install -y nodejs build-essential

Oracle Java

add-apt-repository ppa:webupd8team/java
apt-get update
apt-get install oracle-java8-installer

Elasticsearch

wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | apt-key add -
echo "deb http://packages.elastic.co/elasticsearch/2.x/debian stable main" | tee -a /etc/apt/sources.list.d/elasticsearch-2.x.list
apt-get update && apt-get install elasticsearch
update-rc.d elasticsearch defaults
service elasticsearch start

RVM

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
\curl -sSL https://get.rvm.io | bash -s stable

Ruby 2.4.0

rvm requirements
echo "ruby_url=https://cache.ruby-china.org/pub/ruby" > ~/.rvm/user/db # if need mirror.
rvm install 2.4.0 --disable-binary
rvm use 2.4.0 --default
echo "gem: --no-document" >> /etc/gemrc
echo "gem: --no-document" >> ~/.gemrc
gem sources --add https://gems.ruby-china.org/ --remove https://rubygems.org/ # if need mirror
gem install bundler

Nginx

add-apt-repository ppa:nginx/stable
apt-get update && apt-get install nginx

Create deploy user

sudo adduser \
--system \
--shell /bin/bash \
--gecos 'deploy user' \
--group \
--disabled-password \
--home /data/www deploy
mkdir /data/www/.ssh
cp ~/.ssh/authorized_keys /data/www/.ssh/
chown deploy:deploy -R .ssh/

Prepare setting files while doing cap deploy

cap production deploy
cp config/secrets.yml.default config/secrets.yml
cp config/database.yml.default config/database.yml
cp config/config.yml.default config/config.yml
cp config/redis.yml.default config/redis.yml
cp config/puma.example.rb config/puma-web.rb
cp config/elasticsearch.yml.default config/elasticsearch.yml

Change as necessory

vi /data/www/ruby-china/shared/config/database.yml
vi /data/www/ruby-china/shared/config/config.yml
vi /data/www/ruby-china/shared/config/secrets.yml

notice SECRET_KEY_BASE can get from rails secret

Create DB

sudo su - postgres
psql
CREATE ROLE deploy;
ALTER ROLE deploy LOGIN;
CREATE DATABASE ftghub_prod WITH ENCODING='UTF8' OWNER='deploy';
vi /etc/postgresql/9.6/main/pg_hba.conf
# TYPE DATABASE USER ADDRESS METHOD
local ftghub_prod deploy peer
service postgresql restart

Deploy

cap production deploy

Mapping nginx to puma

upstream ruby_china_backend {
# This is the socket we configured in unicorn.rb
server unix:///data/www/ruby-china/shared/tmp/sockets/puma.sock fail_timeout=0;
keepalive 3;
}
server {
listen 80;
server_name ftghub.com;
root /www/ruby-china/current/public;
gzip_static on; #to serve pre-gzipped version
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html break;
}
location ~ (/assets|/system|/avatar.png|/favicon.ico|/*.txt) {
access_log off;
expires 14d;
gzip_static on;
add_header Cache-Control public;
}
location / {
if ($host != 'ftghub.com') {
rewrite ^/(.*)$ http://ftghub/$1 permanent;
}
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_buffering on;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header X-Forwarded-Proto https;
proxy_pass http://ruby_china_backend;
gzip on;
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /www/ruby-china/current/public;
}
}

Running DB seed

bundle exec rake db:seed RAILS_ENV=production

Reindex ElasticSearch

Must ensure those model at least having one record each.

rake environment elasticsearch:import:model CLASS=Page FORCE=y RAILS_ENV=production
rake environment elasticsearch:import:model CLASS=Topic FORCE=y RAILS_ENV=production
rake environment elasticsearch:import:model CLASS=User FORCE=y RAILS_ENV=production

Monitor Sidekiq Service With Monit & Capistrano

Permalink

Sidekiq as a background processing facility in Rails, usually more and more important during business grow, but it’s relative quite possible failed to startup due to there is some code error, so here is my solution to monitor it.

I choose the monit instead of god beause monit is written in C and very small memory footprint. The capistrano-sidekiq also having a out of box support, but when you combine with monit, capistrano, sidekiq it still trick, so I feel writing a blog is worth.

Install monit

yum install monit # assure using CentOS

Configure monit

vi /etc/monitrc:

set daemon 60
set mailserver localhost
set mail-format {
from: monit@your-domain.com
subject: monit alert -- $EVENT $SERVICE
message: $EVENT Service $SERVICE
Hi all people emails address;
Event happen at: $DATE
Action: $ACTION
Host: $HOST
Description: $DESCRIPTION
Your faithful employee,
Monit (robot)
Hint: when resolved, please replay all.
}
set alert receiver1@mail.com
set alert receiver2@mail.com

Configure in capistrano-sidekiq

At config/deploy/production.rb

set :sidekiq_monit_conf_dir, '/etc/monit.d' # for CentOS folder
set :sidekiq_monit_use_sudo, false

At Capfile

require 'capistrano/sidekiq'
require 'capistrano/sidekiq/monit' # to require monit tasks

Enable alert by customize the monit template.

rails g capistrano:sidekiq:monit:template

and change as below:

# Monit configuration for Sidekiq : <%= fetch(:application) %>
<% processes_pids.each_with_index do |pid_file, idx| %>
check process <%= sidekiq_service_name(idx) %> with pidfile "<%= pid_file %>"
if changed pid then alert
start program = "/bin/su - <%= @role.user %> -c 'cd <%= current_path %> && <%= SSHKit.config.command_map[:sidekiq] %> <%= sidekiq_config %> --index <%= idx %> --pidfile <%= pid_file %> --environment <%= fetch(:sidekiq_env) %> <%= sidekiq_concurrency %> <%= sidekiq_logfile %> <%= sidekiq_queues %> <%= sidekiq_options_per_process[idx] %> -d'" with timeout 30 seconds
stop program = "/bin/su - <%= @role.user %> -c 'cd <%= current_path %> && <%= SSHKit.config.command_map[:sidekiqctl] %> stop <%= pid_file %>'" with timeout <%= fetch(:sidekiq_timeout).to_i + 10 %> seconds
group <%= fetch(:sidekiq_monit_group, fetch(:application)) %>-sidekiq
<% end %>

Deploya to generate minit config

cap production deploy

After that checking if /etc/monit.d having sidekiq conf file. or running cap sidekiq:monit:config to generate config.

Running monit at production

monit

To test the mail, using monit reload, if still can not received the mail, checking my postfix blog.

Building a Postfix Satellite SMTP Service to Avoid Active Mailer Run in Aliyun

Permalink

Some situation is better avoid Sidekiq like reporting the exception mail, so it’s always the best practices to building a Postfix Satellite SMTP service instead.

Install Postfix

yum install -y postfix cyrus-sasl-plain # cyrus-sasl-plain is no need for Ubuntu
yum erase -y sendmail* # In case Sendmail is installed

Configure

Similar to gmail or SendGrid, every mail service vendor offer different combination of setting, below is only apply to AliYun

/etc/postfix/main.cf change

# comment out below
#inet_interfaces = localhost
relayhost = [smtp.mxhichina.com]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_tls_CApath = /etc/ssl/certs
smtp_use_tls = yes
smtp_generic_maps = hash:/etc/postfix/generic
mynetworks = 127.0.0.0/8
mydestination =

Create /etc/postfix/sasl_passwd

[smtp.mxhichina.com]:587 no-reply@domain.com:PASSWORD

Create /etc/postfix/generic

root@staging.domain.com no-reply@domain.com

Create a database file

postmap /etc/postfix/sasl_passwd /etc/postfix/generic

Prevent non-root access:

chmod 0600 /etc/postfix/sasl_passwd /etc/postfix/sasl_passwd.db /etc/postfix/generic /etc/postfix/generic.db

Restart Postfix service

/etc/init.d/postfix restart # or service postfix restart

Debug

It’s very common that postfix will failed to sent mail at first time, so add below to main.cf.

debug_peer_level = 3
debug_peer_list = smtp.mxhichina.com

And open another console to testing via:

echo "body of your email" | mail -s 'mail subject from console' -r 'no-reply@domain.com' guochunzhong@domain.com

You may also need postconf -n to review postfix configuration or postconf -d to review all setting.

YOu can also monitor the postfix activity via tail -f /var/log/syslog.

View Rails API and Guides in Localhost

Permalink

Dash 3 switch to the subscription mode, if you just want to see the Rails document, in fact, it’s relative easier to build in your own box.

Building Rails API Document

cd ~/git/oss/
git clone https://github.com/rails/rails
cd rails/
bundler install
rake rdoc

Building Rails Guides

cd guides/
rake guides:generate:html

Enable Apache

sudo apachectl start

Link Rails documents

cd /Library/WebServer/Documents/
sudo ln -s /Users/guochunzhong/git/oss/rails/doc/rdoc/ rails_api
sudo ln -s /Users/guochunzhong/git/oss/rails/guides/output/ rails_guides

View in localhost

visit Rails API at http://localhost/rails_api/

visit Rails Guides at http://localhost/rails_guides/

Installation Log on Ubuntu 14.04 LTS on vSphere 5.5

Permalink

My log to build a Ubuntu server to fit my software stack usage, R & Ruby.

Install SSH

Upgrade Ubuntu and enable SSH server
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install open-vm-tools
sudo apt-get install openssh-server

Install JDK

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update && sudo apt-get install oracle-java7-installer

Install rvm

sudo su - # switch to root if not yet
apt-get install curl
curl -L get.rvm.io | bash -s master # install rvm
logout
sudo su -
rvm requirements # to install all depends

Install Ruby

rvm install ruby-2.2.3
gem source --remove https://rubygems.org/
gem source --add https://ruby.taobao.org/
gem update --system
apt-get install libgmp3-dev
echo "gem: --no-document" >> /etc/gemrc
gem update
gem install bundler

Install Oracle Client

apt-get install unrar
apt-get install libaio-dev # must need if you want to use sqlplus
mkdir -p /opt/oracle && cd /opt/oracle
unrar x /home/eric/instantclient-all-linux.x64-11.2.0.4.0.rar .
chmod +x instantclient_11_2/sqlplus
ln -s /opt/oracle/instantclient_11_2/libclntsh.so.11.1 /opt/oracle/instantclient_11_2/libclntsh.so
ln -s /opt/oracle/instantclient_11_2/sqlplus /usr/local/bin/sqlplus
echo "/opt/oracle/instantclient_11_2" >> /etc/ld.so.conf.d/oracle_instantclient.conf
# append below line to /etc/enviroment file
ORACLE_BASE=/opt/oracle
ORACLE_HOME=/opt/oracle/instantclient_11_2
TNS_ADMIN=/opt/oracle/network/admin
LD_LIBRARY_PATH=/opt/oracle/instantclient_11_2
OCI_HOME=/opt/oracle/instantclient_11_2
OCI_LIB=/opt/oracle/instantclient_11_2
OCI_LIB_DIR=/opt/oracle/instantclient_11_2
OCI_INCLUDE_DIR=/opt/oracle/instantclient_11_2/sdk/include
NLS_LANG=AMERICAN_AMERICA.AL32UTF8
rvmsudo_secure_path=1

Install Oracle gems ruby-oci8

update the library and install ruby-oci8
reboot
sudo su -
ldconfig
mkdir -p network/admin
mv /home/eric/tnsnames.ora .
gem install ruby-oci8 # should be no any error here :-)

Install R

add R source
sudo vi /etc/apt/sources.list
# append below line to end of sources.list
# you can view mirror at http://cran.r-project.org/mirrors.html
deb http://cran.ism.ac.jp/bin/linux/ubuntu trusty/
prepare to install R
apt-get update
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9
# replace your NO_PUBKEY error above, if pause/block port, using 80 as below line
gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys E084DAB9
gpg -a --export E084DAB9 | sudo apt-key add -
apt-get update # make sure no error here
install R
apt-get install r-base

Install ROracle

manual install the ROracle
wget http://cran.r-project.org/src/contrib/DBI_0.3.1.tar.gz
R CMD INSTALL DBI_0.3.1.tar.gz
wget http://cran.r-project.org/src/contrib/ROracle_1.2-1.tar.gz
R CMD INSTALL --configure-args='--with-oci-inc=/opt/oracle/instantclient_11_2/sdk/include --with-oci-lib=/opt/oracle/instantclient_11_2' ROracle_1.2-1.tar.gz

Install Passenger

Just following Passenger documentation

install phusionpassenger
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 561F9B9CAC40B2F7
sudo apt-get install -y apt-transport-https ca-certificates
sudo sh -c 'echo deb https://oss-binaries.phusionpassenger.com/apt/passenger trusty main > /etc/apt/sources.list.d/passenger.list'
sudo apt-get update
sudo apt-get install -y nginx-extras passenger
vilink
passenger_root /usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini;
passenger_ruby /usr/local/rvm/gems/ruby-2.2.3/wrappers/ruby;
passenger_max_pool_size 30;

Install R studio server

Install R studio server

sudo apt-get install gdebi-core
wget https://download2.rstudio.org/rstudio-server-0.99.473-amd64.deb
sudo gdebi rstudio-server-0.99.473-amd64.deb
vilink
# Server Configuration File
www-address=127.0.0.1
vilink
server {
listen 80;
server_name cvprstudio;
server_name cvprstudio.sandisk.com;
client_max_body_size 10m;
location / {
proxy_pass http://127.0.0.1:8787;
proxy_redirect http://127.0.0.1:8787/ $scheme://$host/;
}
}
cd /etc/nginx/sites-enabled/
ln -s /etc/nginx/sites-available/cvprstudio cvprstudio
service nginx restart

Install RStudio Shiny Server

Install RStudio Shiny Server

run as root
R -e "install.packages('shiny', repos='https://cran.rstudio.com/')"
wget https://download3.rstudio.org/ubuntu-12.04/x86_64/shiny-server-1.4.0.721-amd64.deb
gdebi shiny-server-1.4.0.721-amd64.deb
vilink
server {
listen 80;
server_name shinyapp;
server_name shinyapp.sandisk.com;
location / {
proxy_pass http://127.0.0.1:3838;
proxy_redirect http://127.0.0.1:3838/ $scheme://$host/;
}
}
cd /etc/nginx/sites-enabled/
ln -s /etc/nginx/sites-available/shinyapp shinyapp
service nginx restart

Following patch process in previous blog to re-install the shiny

R CMD INSTALL shiny_0.12.2-server.tar.gz