Running Stable Diffusion v2.1 Base on the Mac Studio and macOS 13 Ventura

May 1st, 2023 • Permalink

Stable Diffusion and MidJourney are the two of the most popular text-to-image models today.. MidJourney need to pay minimal $8 per month but Stable Diffusion is $9 per month but you can freely to run in local.

I have no NVidia GPU, but I have a Base model of Mac Studio, so I would like to try after Apple declared CoreML support Stable Diffusion.

Please make sure you have a good network and have upgraded macOS, aka macOS Ventura before beginning.

brew install miniconda
gh repo clone apple/coremltools
cd coremltools # just clone repo
conda init bash
./scripts/build.sh --python=3.10
xcode-select --install

Then running

python -m python_coreml_stable_diffusion.pipeline --prompt "a photo of an astronaut riding a horse on mars" -i models/coreml-stable-diffusion-v1-4_original_packages -o output --compute-unit CPU_AND_GPU --seed 305
# If not running stable-diffusion-v1-4, the --model-version must be specified.
python -m python_coreml_stable_diffusion.pipeline --prompt "a photo of an astronaut riding a horse on mars" --compute-unit CPU_AND_GPU -o output --seed 1106 -i models/coreml-stable-diffusion-2-1-base_original_packages --model-version stabilityai/stable-diffusion-2-1-base
# Fix the model path if an error is reported.
subl /opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/python_coreml_stable_diffusion/pipeline.py
subl /opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/python_coreml_stable_diffusion/coreml_model.py

Upgrade Brew Installation Python 3 Package

Apr 30th, 2023 • Permalink

I have a long history with python, recently due to I’m starting using cursor.so, I started to give a try again in python, but now in MacOS brew version.

List all installed package:

python3 -m pip list

Setuptools can be updated via pip, without having to re-brew Python:

python3 -m pip install --upgrade setuptools

Pip can be used to upgrade itself via:

python3 -m pip install --upgrade pip

List of all outdated packages:

pip list --outdated

Upgrade package one by one:

python3 -m pip install --upgrade ipython

Install via requirements.txt via proxy

python3 -m pip install -r requirements.txt --proxy socks5://127.0.0.1:6153

2023年的工作小结与计划

Apr 3rd, 2023 • Permalink

之前在2020年定的计划已经过半，由于今年发生了很多重大变化，特别是AI，深度生成技术的突然出现，有必要重新订立一下新的计划。

这里订立计划还是和2020年一样的原则：一，计划不能和世界线的发展发生大的偏差和背离；二，订立的计划必须能够执行。

世界线

世界发生了深刻的变化，IT 的所有领域几乎都发生了停滞，中台的提出者阿里直接拆分了，鸿蒙失败了，唯一的亮点技术 chatGPT 的确可以提高程序员30%的生产力，但是无论从美国还是中国，这种本质上能够代替人类书写代码的技术，只会进一步降低开发成本，无法增加新的需求（除了 AI 技术本身的硬件需求以外）。

所以目前单位的工作核心任务还是降本增效，减少浪费，减少重复开发，确保现有平台得到充分的利用，能够让现有的平台，代码适应新的 AI 时代是工作的重点。

IT技术发展预测

语言

计算机语言和人类语言将发生融合现象，简洁的计算机语言，比如类型可省略的 ruby, javascript，python 会进一步流行，未来人人都是程序员。

新的计算机语言将极难出现，因为 chatGPT 技术不会对这些新的计算机语言进行训练，这就无形中大大提高了新语言的引入门槛。最终计算机语言会分裂为两个大类，高速语言和胶水语言，高速语言有C/C++, Rust，胶水语言有 ruby, javascript，python，如果两方面的特性都需要，那么选Go。

数据库

MySQL 和 PostgreSQL永远是唯二的选择。Oracel 在 M1 推出后的两年还没有支持苹果架构，其他数据库也完全无法同 MySQL 和 PostgreSQL 的人气和成熟度相比较。

跨平台

已经没有跨平台了，现在只有移动端平台和桌面端平台，移动端平台就是微信小程序或者移动端网页，桌面端管理类应用已经全部web化，专业类应用也会逐渐web化，因为Chrome在113版本发布WebGPU，长期看CAD应用也将Web化。操作系统平台已经没有人关注，如果真的有的话，也只会考虑生态与支持的软件能不能允许的更好。目前苹果的生态最生机勃勃，微软的生态存量软件最丰富。鸿蒙已经失败了。

跨平台的意义相比 Java 时代，意义已经很小了，web 平台本身就是跨平台的，而各个平台也同样可以通过一系列服务暴露平台本身的特有服务，所以未来没有操作系统之争，更多的是一种融合。（美国Windows的桌面市场占用了跌倒了57%）

Web 平台

随着 chatGPT 完全使用 web 平台发布，随着Chrome 的能力越来越多（包括通知，驱动蓝牙硬件等等），Web 平台的重要性进一步提高。

但是新的 Web 开发框架不会再大规模出现了，因为新的框架 chatGPT 无法辅助编程，所以无法推广，这个情况和新语言类似。

人工智能

AI 会进入高速发展期，智能与智能的自动化应用和结合是重点，需要重点关注智能的落地和应用。

Serverless

越来越多的人发现，云计算的成本比自建机房高，现在已经开始了下云运动，Serverless更多的成为一种软件架构上的概念，而不是软件发布的概念。

DevOps / 运维自动化

自动化运维将只留下换硬盘的工人。Google 已经开始对 SRE 进行裁员了。

数据平台/中台

概念已经死了，阿里拆封了，搞中台的 N+1 了。数据平台也是个伪概念，只有数据是真的。

已经完成的工作小结

经过 4 年开发，基本各种需求都或多或少的建构了信息管理平台。当然这些系统由于建构时间有先后，建构人员有不同，质量良莠不齐，运营成本差异大，用户满意度也各不相同。我主力制作的基于 Rails 全栈框架的，在质量和运营成本上都基本做到了最优。用户满意度方面，本质上用户还是希望自己做的越少越好，系统做的越多越好，所以过去几年，满意度提升缓慢。由于现在我们代码的质量基础都很优异，所以还是留下了极大的改进空间，所以随后几年可以做的事很多。

经过 4 年基本建立了两个数据仓库，一个 Oracle 的，一个 MySQL 的。各个业务线的数据都能在单位内部，特别是 IT 部内部流通，MySQL的仓库性能也基本不用考虑太多，是今后的重点发展方向。

未来一年计划

进一步推动 API 互通

目前单位针对各个业务口，需求口的系统基本已经建立了，但是还没有相互调用，相互互通，在保证安全的前提下，通过API相互调用还需要加强。

探索引入深度生成技术

深度生成技术包括语言文字生成，图片生成等。在帮助用户减少工作量方面，大有可为。

逐步提升用户体验

随着前端开发技术的发展，用户将越来越挑剔，前端界面也将越来越复杂，单个页面所能做的事情也会越来越多，在保持低成本开发的同时，要提升用户的体验，就必须开发复杂的用户界面，需要在 Rails 做后端的基础上，逐步引入 Turbo 和 React 。

加强团队合作

谨慎评估员工的表现和贡献，提高员工工作效率和满意度，在需求清晰的情况下，尽量不打扰开发人员，帮助他们始终高效开发，同时也通过Gitlab统一代码管理平台，进行适度监控，适当鞭策开发。

Fix Yarn Packaging Signature Invalid - EXPKEYSIG 23E7166788B63E1E Yarn Packaging

Mar 4th, 2023 • Permalink

When running apt-get update at Ubuntu 20.04.5 LTS, got below errors:

W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://dl.yarnpkg.com/debian stable InRelease: The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging <yarn@dan.cx>
W: Failed to fetch https://dl.yarnpkg.com/debian/dists/stable/InRelease  The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging <yarn@dan.cx>
W: Some index files failed to download. They have been ignored, or old ones used instead.

To resolve just running below command.

export YARNKEY=yarn-keyring.gpg
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo gpg --dearmour -o /usr/share/keyrings/$YARNKEY
echo "deb [signed-by=/usr/share/keyrings/$YARNKEY] https://dl.yarnpkg.com/debian stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

Deploy New Rails 7 and Ruby 3.2 App in Ubuntu 20.04

Feb 15th, 2023 • Permalink

Create new user

adduser pp_vendor
sudo su - pp_vendor
mkdir .ssh
chmod 700 .ssh
vi .ssh/authorized_keys # and paste your public key
chmod 600 .ssh/authorized_keys

Enable new user as sudo

sudo su -
cd /etc/sudoers.d/
echo "pp_vendor ALL=(ALL) NOPASSWD:ALL" > 85-pp_vendor-user

Install Rust

sudo apt install rustc
rustc --version # here is rustc 1.61.0

Install rbenv and Ruby 3.2.1

sudo apt-get install libyaml-devel
sudo apt install rbenv
sudo su - pp_vendor
mkdir -p "$(rbenv root)"/plugins
git clone https://git.thape.com.cn/rails/ruby-build.git "$(rbenv root)"/plugins/ruby-build
git clone https://git.thape.com.cn/rails/rbenv-china-mirror.git "$(rbenv root)"/plugins/rbenv-china-mirror
rbenv install 3.2.1
rbenv global 3.2.1
echo "gem: --no-document" > ~/.gemrc
eval "$(rbenv init -)" >> ~/.bash_profile # or past the `rbenv init -`
rbenv shell 3.2.1

Create MySQL DB user

mysql -u root -p
CREATE DATABASE pp_prod character set UTF8mb4 collate utf8mb4_bin;
CREATE USER 'pp_vendor'@'%' IDENTIFIED BY 'pp_vendor_password';
GRANT ALL ON pp_vendor.* TO 'pp_vendor'@'%';
FLUSH PRIVILEGES;

Link rbenv to make capistrano works

mkdir -p ~/.rbenv/bin
cd ~/.rbenv/bin
ln -s /usr/bin/rbenv rbenv

create deploy folder

cd /var/www
sudo mkdir pp_vendor
sudo chown pp_vendor:pp_vendor pp_vendor/
echo "machine git.thape.com.cn login Eric-Guo password token_of_personal" >> ~/.netrc

Manually Do Upload in ActiveStorage in Rails Console

Feb 7th, 2023 • Permalink

Upload file is at local:

rails console

a=Article.last
a.download_file.attach(io: File.open("/home/deployer/Manual.docx"), filename: "使用手册区.docx")

Upload file is at remote:

rails console

file = URI.open(remote_file_full_url)
a=Article.last
a.download_file.attach(io: file, filename: remote_file_name)

Some Pro User MacOS Settings

Jan 23rd, 2023 • Permalink

Lock the fix area of the Dock.

defaults write com.apple.dock contents-immutable -bool yes
defaults write com.apple.dock size-immutable -bool yes
defaults write com.apple.dock position-immutable -bool yes
killall Dock

Setting recent application number, default is 3.

defaults write com.apple.dock show-recent-count -int 5;
killall Dock

More modern macOS can not stop upgrade from setting, but still can clean the red dot.

defaults write com.apple.systempreferences AttentionPrefBundleIDs 0

Install Open Project in a Shard Rocky Linux Server

Jan 18th, 2023 • Permalink

The target server has the Rocky Linux installed and running the homeland forum already. I already downgrade the node.js from 18 to 16 to meet the open project needs.

Setup open_project user account

adduser open_project
cd /etc/sudoers.d/
echo "open_project ALL=(ALL) NOPASSWD:ALL" > 30-open_project-user
sudo su - open_project
mkdir .ssh
chmod 700 .ssh
vi .ssh/authorized_keys # and paste your public key
chmod 600 .ssh/authorized_keys

Install rbenv and ruby-build

whoami # should run as a open_project
git clone https://git.thape.com.cn/rails/rbenv.git .rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile
~/.rbenv/bin/rbenv init # also edit ~/.bash_profile
# As an rbenv plugin
mkdir -p "$(rbenv root)"/plugins
git clone https://git.thape.com.cn/rails/ruby-build.git "$(rbenv root)"/plugins/ruby-build
git clone https://git.thape.com.cn/rails/rbenv-china-mirror.git "$(rbenv root)"/plugins/rbenv-china-mirror

Install Ruby 3.2.0

Ruby 3.2 need Rust to build JIT.

yum --enablerepo=powertools install libyaml-devel
yum install -y rust

rbenv install -l
rbenv install 3.2.0
rbenv global 3.2.0
rbenv shell 3.2.0
echo "gem: --no-document" > ~/.gemrc
gem update --system

Prepare the capistrano deploy folder

whoami # should run as a open_project
cd /var/www
sudo mkdir open_project
sudo chown open_project:open_project open_project/

Create postgresql db user and import DB

sudo su - postgres
createuser open_project
psql
ALTER ROLE open_project LOGIN;
CREATE DATABASE open_project_prod WITH ENCODING='UTF8' OWNER=open_project;
logout
psql -d open_project_prod -f open_project_db.sql

Using mirror when deploy

Run in the release rails root folder

bundle config mirror.https://rubygems.org https://gems.ruby-china.com

Setting the open project settings

/etc/environment

OPENPROJECT_EDITION=bim
OPENPROJECT_APP__TITLE=Open Project
OPENPROJECT_HOST__NAME=op.cloud-mes.com

~/.config/systemd/user/open_project_puma.service

[Unit]
Description=Puma HTTP Server for open_project
After=syslog.target network.target
 
[Service]
Type=simple
WatchdogSec=10
Environment="OPENPROJECT_EDITION=bim"
WorkingDirectory=/var/www/open_project/current
ExecStart=/home/open_project/.rbenv/bin/rbenv exec bundle exec puma -C config/puma_prod.rb -e production
ExecReload=/bin/kill -USR1 $MAINPID
 
# if we crash, restart
RestartSec=5
Restart=on-failure
 
StandardOutput=append:/var/www/open_project/shared/log/puma.log
StandardError=append:/var/www/open_project/shared/log/puma.log
SyslogIdentifier=open_project_puma_staging
 
[Install]
WantedBy=default.target

systemctl --user daemon-reload
bundle exec rake openproject:plugins:register_frontend
bundle exec rake i18n:js:export
bundle exec rake db:seed

Downgrade Node 18 to 16 in Rocky Linux

Jan 17th, 2023 • Permalink

When start developing open project, I found it only support Node.js 16 and confirm Node.js 18 not working currently, so I need downgrade the node version in a Rocky Linux server to build a staging environment and here is how:

Uninstall node 18

dnf remove nodejs
cd /etc/yum.repos.d/
rm nodesource-el8.repo
yum update

Install node 16

curl -fsSL https://rpm.nodesource.com/setup_16.x | bash -
dnf install -y nodejs
node --version
dnf install -y yarn

← Older Blog Archives