Eric Guo's blog.cloud-mes.com

in HTML5, JavaScript, Ruby & Rails, Python, and Cloud MES!

Ruby Conf China 2017 Notes

Permalink

第一场 m-ruby on nginx from strikely

Performance Result

目测删除了对m-ruby的结果,比如OpenResty的测试。

XSS via blog using m-ruby

可以通过nginx层改json的格式,通过mruby语法,性能未知。

Real usage in strikely

Image compress, jwt token verify

Sample setup code

https://github.com/tylerdiaz/ngx-ruby

Limitations

1.9.3 syntax only and can not require files and existing ruby gems.

第二场 Ethereum on Ruby

Rust和Ruby很有渊源,因为它们开头都是Ru。。。

区块链组成:加密算法、P2P网络、共识算法、可证实的数据结构(区块链 Authenticated Data Structure)。

Contract Account是的机器在历史上第一次财务独立,可以代替一切中间人的角色。

EVM Ethereum Virtual Machine 计费力度可以到指令。

交易一般在3秒内传遍世界,EVM跑交易,15秒出一个区块块。

Solidity, Viper(类Python,类型更安全), Bamboo(做形式化证明)

Precompile Contract 预编译的合约用于加速处理,都是加密相关的功能。

第三场 Ruby异步编程奥德赛

全程无尿点,强烈推荐。

第四场 Erlang开发web框架

浪费一个小时。。

第五场 Ruby Web实时通讯方案剖析 侯俊杰

NChan方案。

第六场 Exploring ActiveRecord

第七场 Docker 发布

第八场 Ruby-Packer

Based on SquashFS

第九场 Elixir

第十场 金数据鉴黄

tesseract-ocr 字符识别

第十一场 mobx-ruby

New EC2 Box on Amazon Linux Install Log

Permalink

Install postgresql 9.6

sudo yum update
# https://yum.postgresql.org/repopackages.php#pg96
sudo yum install https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-6-x86_64/pgdg-ami201503-96-9.6-2.noarch.rpm
sudo yum install -y postgresql96 postgresql96-server postgresql96-libs postgresql96-contrib
sudo /etc/init.d/postgresql-9.6 initdb
sudo /etc/init.d/postgresql-9.6 start
sudo chkconfig postgresql-9.6 on
sudo -u postgres -i psql -c 'SELECT version();'

Install nginx

sudo yum install -y nginx
sudo chkconfig nginx on

Install redis

See https://medium.com/@andrewcbass/install-redis-v3-2-on-aws-ec2-instance-93259d40a3ce

Install node.js

curl --silent --location https://rpm.nodesource.com/setup_6.x | bash -
yum install -y gcc-c++ make
yum install -y nodejs

Install Go

wget https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz
echo "export PATH=$PATH:/usr/local/go/bin" > /etc/profile.d/go-lang.sh

Install shadowsocks-go

sudo yum install -y git
go get github.com/shadowsocks/shadowsocks-go/cmd/shadowsocks-server
wget -4qO- "http://whatismyip.akamai.com/" # record public IP

Build config.json

{
"server":"127.0.0.1",
"server_port":8388,
"local_port":1080,
"password":"barfoo!",
"method": "aes-128-cfb-auth",
"timeout":600
}

cd ~/go/bin && nohup ./shadowsocks-server &

Install Mosh

sudo yum --enablerepo=epel install -y mosh

Using POW and Byebug Together

Permalink

Some rails application, especially in SaSS category will behave differently based on the different subdomain. In that case, pow is necessary, as it makes simulate subdomain much easier compare with edit based on /etc/hosts.

Setup the pow relative easy:

brew install pow
mkdir -p ~/Library/Application\ Support/Pow/Hosts
ln -s ~/Library/Application\ Support/Pow/Hosts ~/.pow
pow --install-local
launchctl load -w ~/Library/LaunchAgents/cx.pow.powd.plist

Then you can link required rails application via:

cd ~/.pow
ln -s /Users/guochunzhong/git/product_hunt/ product-hunt

And finally, using http://product-hunt.dev to access rails application without using rails s.

Because now there is no terminal to running rails s, so you now have to use tail -f log/development.log to monitor, it’s not a big deal, in fact, more flexible as it enables you Ctrl+C any time and leaves the interesting log in the terminal to read.

The big issue is now you are no access the Byebug as even Byebug is paused, we have no access to it, so must using remote mode of byebug.

Luckily, it’s still not too hard, just following product_hunt commit will do the job.

Explore Rails 5.1 New Feature via an Simple Project

Permalink

Rails 5.1即将发布,虽然同Rails 5相比,这只是一个0.1的小版本,但我看来,Rails 5.1的新的功能却非常重要:在JavaScript世界建构和模块管理工具突飞猛进了3~4年后(grunt, gulp, browserify, npm, webpack, yarn),一向喜欢稳定的Rubyist们(喜欢折腾的都跑去javascript, go和Elixir了。。),在大神DHH和女神Liceth以及其他众神的帮助下,迎来了和node.js国的最终和解:YarnWebpack已经正式成为了主厨推荐

本文和其他介绍Rails 5.1 Whatsnew稍有不同的地方在于,我试图通过一个新的项目探索Rails 5.1的特点。

product hunt是一个功能简单的Rails应用,仅仅实现了产品的维护,但在功能方面,探索了yarn下对新javascript框架的使用,还有集成测试。

具体来说,使用了比较小众的milligram CSS框架,输入自动提示使用了零依赖的awesomplete,使用基于Chrome的集成测试。

Yarn

Yarn同npm相比,优点很多,yml格式的yarn.lock文件显示依赖关系清晰易读,安装速度快,完全不需要再使用asset pipe line对已有的javascript模块进行二次封装,省事省力,省开新gem。环境准备也相当容易,在国内(必须)用淘宝源即可。

brew install yarn
yarn config set registry 'https://registry.npm.taobao.org'

设置完毕后,就可以直接使用yarn添加依赖了。

yarn add milligram

使用CSS前端框架,直接引用即可:

/* app/assets/stylesheets/application.css */
*= require milligram
// app/assets/stylesheets/milligram.sass
@import milligram/src/Color
@import milligram/src/Base
@import milligram/src/Button

使用javascript库多做一步:

// app/assets/config/manifest.js
//= link awesomplete/awesomplete.js
<%# app/views/products/_form.html.erb %>
<%= javascript_include_tag "awesomplete", async: true -%>

Webpack

Webpack是可选的,如果是majestic monolith的Rails应用的话,肯定不会用,但是现代的前后端分离风潮下,至少Rails 5.1让这个前后端分离变得无比容易,比之前的react_on_rails方案容易好多。

jQuery的依赖去除

由于Javascript一众MVC框架的崛起,Rails 5.1在前端方案上给予了开发者更多的选择权,我试了在没有jquery的情况下,使用0 payloading的vanilla-js开发,虽然时不时还需要查一下对应jQuery的用法,但这样做还是值得的,微信小程序的卖点就是即用即走,但如果能将网页做到轻量,何尝不是即用即走?况且网页发布还不用走审核流程。

在Rails下使用vanilla js其实也不是什么都没得用,rails-ujs始终是存在的,所以还是可以直接使用Rails.ajax方法来发起远程调用:

window.addEventListener('input', function (e) {
if (e.target.id == "product_name") {
Rails.ajax({
type:'GET',
url: e.target.getAttribute('data-url'),
dataType: 'script'
});
}
}, false);

System Test

Rails 5.1对集成测试也有了官方方案,现在出厂即可跑集成测试。集成测试在测试自动提示这样的特性的时候还是很方便的,官方出厂的默认配置基于selenium-webdriver驱动的Chrome,使用之前需要安装驱动:

brew install chromedriver
require 'application_system_test_case'
class NewProductTest < ApplicationSystemTestCase
test 'create a new product' do
visit '/products/new'
fill_in 'product_name', with: 'L'
page.has_selector?('ul > li > mark')
fill_in 'product_name', with: 'Le Wagon'
page.has_no_selector?('ul > li > mark')
fill_in 'product_tagline', with: 'Change your life: Learn to code'
click_button 'Create Product'
# Should be redirected to Home with new product
assert_equal product_path(Product.last), page.current_path
assert page.has_content?('Change your life: Learn to code')
end
end

这里第一次在product_name填入‘L',应该有自动完成的提示出现,完全输入后,则没有,这些都可以通过Capybara的浏览器DSL做检测,从而在集成测试中一并测试好,相比之前的单个controller测试,效率提高不少。

其他特性。

其他的特性:Encrypted secrets、Parameterized mailers、Direct & resolved routes等,参见Rails 5.1 Release notes,演示项目中没有包括这些功能,这里就略过了。

本文不足

测试方案还想使用phantomjs和poltergeist,但是始终没有成功,有兴趣的同学可以尝试并提Pull Request。

Design Resource & Tips

Permalink

A list of design resource from lewagon

UI do’s and don'ts

https://goodui.org/

Pictures

Colors

Background gradient

UI gradients

Fonts - Google fonts

Make your shopping on Google fonts.

For main typo

  • Open-Sans

For headers

  • Raleway
  • Montserrat
  • Varela Round
  • Museo

Fonts - Hack on Github

Look for .ttf/.eot/.woff/etc.. files on Github.

  • Avenir
  • Proxima Nova
  • Gotham Rounded
  • Brandon

Icons

HTML/CSS snippets

Inspiration

在Macbook Pro 2016中原生安装tensorflow 1.0

Permalink

前几天的TensorFlow开发者峰会宣布的TensorFlow 版本达到1.0,再加上过去一年人工智能,机器学习非常非常热,虽然作为一名Ruby程序员,但形势所迫,又不得不捡起了Python,准备稍稍了解一下。

TensorFlow的安装官网写的很清晰,我选择的是virtualenv的安装,安装的python版本是brew的版本。

brew install python
sudo easy_install pip
sudo pip install --upgrade virtualenv
virtualenv --system-site-packages ~/tensorflow
source ~/tensorflow/bin/activate
pip install --upgrade tensorflow

安装完成后每次运行需要先运行source ~/tensorflow/bin/activate,嫌麻烦可以把这句加到.bash_profile中。

echo "source ~/tensorflow/bin/activate" >> ~/.bash_profile

安装完成后可以跑一下tensorflow版的“Hello Word”验证一下安装是否正确,需要先进入python的交互环境:

python

然后运行:

import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))

问题来了,一堆CPU指令没有优化,Macbook Pro 2016已经没有核弹卡了,只能跑在CPU版上了,如果CPU再不优化,感觉已经很难愉快的玩下去了,所以只能选择从源码安装了。

源码安装就很烦了,下载源吗就非常费网络,还必须FQ,编译也需要至少25分钟时间,不过亲爱的读者,我已经帮你们做好了,现在可以使用我的安装包直接升级到原生CPU优化版:

pip install --upgrade http://baye-deploy.oss-cn-shanghai.aliyuncs.com/tensorflow-1.1.0-cp27-cp27m-macosx_10_12_x86_64.whl

巴爷供销社,一个生产级别的微信小应用商城开源实现

Permalink

今天是微信小应用发布的日子,很高兴巴爷科技(上海)有限公司能作为第一批微信小应用的Rails厂商,能为Rails社区提供一个开源的参考实现。(包括后端)

微信小应用的介绍文章已经很多了,快速的开发体验,iOS/Android不必分别开发俩次都是很大的优点。但同时,腾讯微信也不是上次的公众号发布时的微信了,7亿的用户,近乎100%的中国市场,所以腾讯变的非常非常小心谨慎,所以也不要指望能像原来的公众号,能从腾讯分到近乎免费的流量红利。

微信小程序限制非常之多:微信小程序二维码只能通过扫一扫进入,不能通过图片扫描;微信小程序没有应用商店,只能通过发现,小程序的的搜索功能寻找;更令人发指的是,非合作伙伴的话,还必须搜索全名才行,也就是说,只有腾讯投资的京东,才能搜索“京东”找到“京东购物”,像巴爷科技,搜索“巴爷”是无法得到任何结果的,必须搜索“巴爷供销社”。

微信小应用在开发方面,微信基本保持了和服务号近似的接口,例如微信支付的对接。但也有不同的地方,例如敏感数据的加解密等。另外由于我们开始开发等时间较早,持续到现在,很多API不断发生改变,这也确实给我们的开发造成了一些困扰。

微信小应用递交审核方面,由于我们不是内测的200个名单中的公司,而是作为第二批公测用户,递交审核的周期和苹果差不多,一次审核周期在1~2天,甚至周末也有人审核。但不好的地方在于,拒的话,只会告诉你一个问题,而且一旦发现应用有bug,有和应用说明中没写的功能,超出微信小应用所选分类的功能,都会被拒。所以如果你的应用如果想快速上线,基本很难。审核期间我们在是否需要ICP证方面被拒了3次,中国的政策很奇怪,理论上只要你有营利性的网上经营行为就必须要ICP证,否则就属于非法经营。但是如果你到电信管理局去申请这个ICP证,办事人员却会以你的公司业务量太小,拒绝你的申请(🐷要养肥了杀?),但好在腾讯在最后一刻,终于在审核上线时,不需要微信小应用递交ICP证了,所以,现在能访问到“巴爷供销社”,还是要感谢腾讯在政府关系方面做出的努力。

微信小程序的github开源地址:https://github.com/bayetech/wechat_mall_applet

后端基于active_model_serializers的参考实现地址:https://github.com/bayetech/wechat_mall_applet_backend

A Rubyist Web Developer Macbook Pro With Touch Bar Installation Notes

Permalink

Just got the Macbook Pro 2016 with touch bar 15’ edition, setting it up takes my one full day. If the China network condition is good, I believe it will take much less time.

I decide the Macbook Pro from scratch instead of using migration assist because I want to keep the system clean. I will install as much application as possible from App Store, then the Homebrew, if both software repository can not find it, install it manually.

Apple store installation is the easiest one, once you log in and you will get all the application you have purchased.

brew are also relative easier, I even using brew to install Chrome and Firefox, via brew cask install google-chrome.

I would like to use percona instead of MySQL because, in China, Aliyun RDS is primarily based on percona, so I favor using the same software in dev env.

But you can not simply install percona directly.

Install mysql, then percona to avoid pitfall

brew install mysql
brew services start mysql
brew services stop mysql
brew remove mysql
brew install percona-server
brew services start percona-server

twitter cli need to rename tt instead of t

gem install t
mv /usr/local/bin/t /usr/local/bin/tt

Nokogiri using system library

brew install libxml2
# If installing directly
gem install nokogiri -- --use-system-libraries \
--with-xml2-include=$(brew --prefix libxml2)/include/libxml2
# If using Bundle
bundle config build.nokogiri --use-system-libraries \
--with-xml2-include=$(brew --prefix libxml2)/include/libxml2
bundle install

proxychains using Shadowsocks

To add Shadowsocks setting for proxychains.

vi /usr/local/Cellar/proxychains-ng/4.11/etc/proxychains.conf

It will finally take 76Gb after installing all my needs, about 33% used, so I believe in buying a 256Gb SSD edition of rMBP should be enough for my next four years usage.

Deploy Ruby-china as a New Forum Software to Aliyun

Permalink

This wiki based on the Ubuntu 14.04.5 LTS Server.

Prerequisites

apt-get update && apt-get upgrade
apt-get dist-upgrade
apt-get install software-properties-common htop curl git

memcached

apt-get install memcached

Redis

add-apt-repository ppa:chris-lea/redis-server
apt-get update && apt-get install redis-server

PostgreSQL

echo "deb http://apt.postgresql.org/pub/repos/apt/ trusty-pgdg main" > /etc/apt/sources.list.d/pgdg.list
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
apt-get update && apt-get install postgresql
apt-get install libpq-dev

imagemagick

apt-get install imagemagick

node.js

curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
apt-get install -y nodejs build-essential

Oracle Java

add-apt-repository ppa:webupd8team/java
apt-get update
apt-get install oracle-java8-installer

Elasticsearch

wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | apt-key add -
echo "deb http://packages.elastic.co/elasticsearch/2.x/debian stable main" | tee -a /etc/apt/sources.list.d/elasticsearch-2.x.list
apt-get update && apt-get install elasticsearch
update-rc.d elasticsearch defaults
service elasticsearch start

RVM

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
\curl -sSL https://get.rvm.io | bash -s stable

Ruby 2.4.0

rvm requirements
echo "ruby_url=https://cache.ruby-china.org/pub/ruby" > ~/.rvm/user/db # if need mirror.
rvm install 2.4.0 --disable-binary
rvm use 2.4.0 --default
echo "gem: --no-document" >> /etc/gemrc
echo "gem: --no-document" >> ~/.gemrc
gem sources --add https://gems.ruby-china.org/ --remove https://rubygems.org/ # if need mirror
gem install bundler

Nginx

add-apt-repository ppa:nginx/stable
apt-get update && apt-get install nginx

Create deploy user

sudo adduser \
--system \
--shell /bin/bash \
--gecos 'deploy user' \
--group \
--disabled-password \
--home /data/www deploy
mkdir /data/www/deploy/.ssh
cp ~/.ssh/authorized_keys /data/www/deploy/.ssh/
chown deploy:deploy -R .ssh/

Prepare setting files while doing cap deploy

cap production deploy
cp config/secrets.yml.default config/secrets.yml
cp config/database.yml.default config/database.yml
cp config/config.yml.default config/config.yml
cp config/redis.yml.default config/redis.yml
cp config/puma.example.rb config/puma-web.rb
cp config/elasticsearch.yml.default config/elasticsearch.yml

Change as necessory

vi /data/www/ruby-china/shared/config/database.yml
vi /data/www/ruby-china/shared/config/config.yml
vi /data/www/ruby-china/shared/config/secrets.yml

notice SECRET_KEY_BASE can get from rails secret

Create DB

sudo su - postgres
psql
CREATE ROLE deploy;
ALTER ROLE deploy LOGIN;
CREATE DATABASE ftghub_prod WITH ENCODING='UTF8' OWNER='deploy';
vi /etc/postgresql/9.6/main/pg_hba.conf
# TYPE DATABASE USER ADDRESS METHOD
local ftghub_prod deploy peer
service postgresql restart

Deploy

cap production deploy

Mapping nginx to puma

upstream ruby_china_backend {
# This is the socket we configured in unicorn.rb
server unix:///data/www/ruby-china/shared/tmp/sockets/puma.sock fail_timeout=0;
keepalive 3;
}
server {
listen 80;
server_name ftghub.com;
root /www/ruby-china/current/public;
gzip_static on; #to serve pre-gzipped version
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html break;
}
location ~ (/assets|/system|/avatar.png|/favicon.ico|/*.txt) {
access_log off;
expires 14d;
gzip_static on;
add_header Cache-Control public;
}
location / {
if ($host != 'ftghub.com') {
rewrite ^/(.*)$ http://ftghub/$1 permanent;
}
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_buffering on;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header X-Forwarded-Proto https;
proxy_pass http://ruby_china_backend;
gzip on;
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /www/ruby-china/current/public;
}
}

Running DB seed

bundle exec rake db:seed RAILS_ENV=production

Reindex ElasticSearch

Must ensure those model at least having one record each.

rake environment elasticsearch:import:model CLASS=Page FORCE=y RAILS_ENV=production
rake environment elasticsearch:import:model CLASS=Topic FORCE=y RAILS_ENV=production
rake environment elasticsearch:import:model CLASS=User FORCE=y RAILS_ENV=production