构建基于 OpenResty + Lua 的短地址服务
前言
曾经,我用 PHP 原生编写了一个轻量级自定义短链接生成的库(GitHub: laijingwu/url-shortener),依赖 vlucas/phpdotenv 和 catfan/medoo 组件。时隔两年,短信营销推广需要在短信内容中放置短链接,而这次我选择了 OpenResty 和 Lua 来完成。
创建数据库表
这里我们使用了 MySQL 作为持久化的存储,因此需要在 MySQL 中创建一个表用于存储短链接和对应长链接等信息。
CREATE TABLE `short_link` (`id` int unsigned NOT NULL AUTO_INCREMENT,`key` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL,`value` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL,`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,`deleted_at` timestamp NULL DEFAULT NULL,PRIMARY KEY (`id`),KEY `idx_key` (`key`),KEY `idx_created_at` (`created_at`) USING BTREE) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci ROW_FORMAT=DYNAMIC COMMENT='短链接';
编写 Lua 脚本
废话不多说,直接上示例代码。示例代码中使用了 MySQL 作为持久化的存储,使用了 OpenResty 自带的 lua_shared_dict 作为缓存,这里可以根据自身的实际需求去更换不同的存储。
local host = os.getenv("APP_URL")local db_host = os.getenv("DATABASE_HOST")local db_port = os.getenv("DATABASE_PORT")local db_database = os.getenv("DATABASE")local db_user = os.getenv("DATABASE_USERNAME")local db_password = os.getenv("DATABASE_PASSWORD")local expire = os.getenv("CACHE_EXPIRED_MINUTES")if host == nil thenhost = "http://127.0.0.1"endif db_host == nil or db_database == nil or db_user == nil or db_password == nil thenngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY)endif db_port == nil thendb_port = 3306endif expire == nil thenexpire = 15endlocal uri = ngx.var.request_uriif not uri or uri == "/" thenngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY)endlocal links = ngx.shared.linkslocal redirect, flags, err = links:get(uri)if not redirect or redirect == nil thenlocal mysql = require "resty.mysql"local db, err = mysql:new()if not db thenngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY)enddb:set_timeout(1000)local ok, err, errno, sqlstate = db:connect{host = db_host,port = db_port,database = db_database,user = db_user,password = db_password,max_packet_size = 1024 * 1024}if not ok thendb:close()ngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY)endsql = "SELECT value FROM short_link WHERE `key` = " .. "\'" .. uri .. "\'" .. " AND deleted_at IS NULL ORDER BY created_at DESC LIMIT 1"local res, err, errno, sqlstate = db:query(sql)if not res or next(res) == nil thendb:close()ngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY)endlocal result = res[1]["value"]if(type(result) ~= "string") thendb:close()ngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY)endif string.match(result, "^https-://[%w+%.]+[/%w+]+.*") ~= nil thenredirect = resultelseif string.match(result, "^%w[%w+%.]+.*") ~= nil thenredirect = "http://" .. resultelseredirect = host .. "/" .. resultendlocal ok, err = links:set(uri, redirect, 60 * expire)if not ok thendb:close()ngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY)enddb:close()endngx.redirect(redirect, ngx.HTTP_MOVED_PERMANENTLY)
准备 OpenResty 配置文件
nginx.conf 文件如下,供参考:
user nobody;worker_processes 2;error_log /var/log/nginx/error.log warn;pid /var/run/nginx.pid;env APP_URL;env DATABASE;env DATABASE_HOST;env DATABASE_PORT;env DATABASE_USERNAME;env DATABASE_PASSWORD;env CACHE_EXPIRED_MINUTES;events {worker_connections 1024;}http {include mime.types;default_type application/octet-stream;log_format main '$remote_addr - $remote_user [$time_local] "$request" ''$status $body_bytes_sent "$http_referer" ''"$http_user_agent" "$http_x_forwarded_for"';access_log /var/log/nginx/access.log main;sendfile on;tcp_nopush on;keepalive_timeout 65;server_names_hash_bucket_size 64;lua_shared_dict links 128m; # 用于分配缓存大小gzip on;gzip_disable "msie6";include /etc/nginx/conf.d/*.conf;}
shortlink.vhost.conf 配置如下:
server {listen 80;listen [::]:80;server_name xxx.com;access_log /var/log/nginx/shortlink.access.log;error_log /var/log/nginx/shortlink.error.log;resolver local=on ipv6=off; # 读取/etc/resolve.conf中默认dns地址resolver_timeout 5s;location / {default_type text/plain;content_by_lua_file /usr/local/openresty/lualib/shortlink.lua;}location = /robots.txt {return 200 "User-Agent: *\nDisallow: ";}location = /probe {return 200 "";}#error_page 404 /404.html;error_page 500 502 503 504 /50x.html;location = /50x.html {root /usr/local/openresty/nginx/html;}location ~ /\.ht {deny all;}}
搭建本地测试环境
本地的测试环境使用 docker 进行搭建,编写好 docker-compose 和相关配置文件后就可以进行测试。
version: '3'services:mysql:container_name: local-mysqlimage: mysql:8.0.20ports:"3306:3306"volumes:$PWD/conf/mysql/conf.d:/etc/mysql/conf.d:ro$PWD/data/mysql:/var/lib/mysqlenvironment:MYSQL_ROOT_PASSWORD: rootMYSQL_DATABASE: appMYSQL_USERNAME: testMYSQL_PASSWORD: testTZ: Asia/Shanghainetworks:localopenresty:container_name: local-openrestyimage: openresty/openresty:centosports:"80:80"volumes:$PWD/log/openresty:/var/log/nginx$PWD/scripts/openresty/shortlink.lua:/usr/local/openresty/lualib/shortlink.lua$PWD/scripts/openresty/nginx.conf:/usr/local/openresty/nginx/conf/nginx.conf:ro$PWD/scripts/openresty/shortlink.conf:/etc/nginx/conf.d/shortlink.conf:roentrypoint:/usr/bin/openresty-gdaemon off;environment:TZ: Asia/ShanghaiAPP_URL: http://127.0.0.1DATABASE_HOST: mysqlDATABASE_PORT: 3306DATABASE: fusionDATABASE_USERNAME: rootDATABASE_PASSWORD: rootCACHE_EXPIRED_MINUTES: 15depends_on:mysqlnetworks:localnetworks:local:
编写部署文件
完成上述工作后,就可以着手开始进行部署工作了,部署文件示例如下。ConfigMap 配置文件如下:
apiVersion: v1kind: ConfigMapmetadata:name: cm-shortlink-proxyannotations:configVersion: CONFIG_VERSIONnamespace: APP_ENVdata:: |nginx 配置): |vhost 配置): |lua 脚本)
Deployment 配置文件如下:
apiVersion: apps/v1kind: Deploymentmetadata:name: dp-shortlink-proxylabels:app: shortlink-proxynamespace: APP_ENVspec:replicas: 1selector:matchLabels:app: shortlink-proxytemplate:metadata:annotations:configVersion: CONFIG_VERSION: SHORTLINK_VERSIONlabels:app: shortlink-proxyspec:imagePullSecrets:name: registry-keycontainers:name: shortlink-proxyimage: SHORTLINK_IMAGE_REPOSITORY:SHORTLINK_VERSIONimagePullPolicy: IfNotPresentenv:name: APP_FRONTEND_URLvalue: "https://APP_FRONTEND_SERVER_NAME"name: DATABASE_HOSTvalueFrom:secretKeyRef:name: APP_ENV-database-secretkey: hostname: DATABASE_PORTvalue: "DB_PORT"name: DATABASEvalue: "DB_DATABASE"name: DATABASE_USERNAMEvalueFrom:secretKeyRef:name: APP_ENV-database-secretkey: usernamename: DATABASE_PASSWORDvalueFrom:secretKeyRef:name: APP_ENV-database-secretkey: passwordname: CACHE_EXPIRED_MINUTESvalue: "SHORTLINK_CACHE_TIMEOUT"ports:name: shortlink-svccontainerPort: 80volumeMounts:name: shortlink-proxy-logsmountPath: /usr/local/nginx/logsname: shortlink-proxy-confmountPath: /usr/local/openresty/nginx/conf/nginx.confsubPath: nginx.confname: shortlink-proxy-confmountPath: /etc/nginx/conf.d/shortlink.confsubPath: shortlink.confname: shortlink-proxy-confmountPath: /usr/local/openresty/lualib/shortlink.luasubPath: shortlink.luaresources:requests:cpu: 100mmemory: 300Milimits:cpu: 2000mmemory: 2000Micommand: ["/usr/bin/openresty"]args: ["-g", "daemon off;"]readinessProbe:httpGet:path: /probeport: 80initialDelaySeconds: 10periodSeconds: 10volumes:name: shortlink-proxy-logsemptyDir: {}name: shortlink-proxy-confconfigMap:name: cm-shortlink-proxy
Service 配置文件如下:
apiVersion: v1kind: Servicemetadata:name: svc-shortlink-proxyannotations:: httplabels:app: shortlink-proxynamespace: APP_ENVspec:type: LoadBalancerports:name: shortlink-svcport: 80protocol: TCPtargetPort: shortlink-svcselector:app: shortlink-proxy
Secret 配置文件如下:
apiVersion: v1kind: Secretmetadata:name: APP_ENV-database-secrettype: OpaquestringData:host: DB_HOSTusername: DB_USERNAMEpassword: DB_PASSWORD
上面的配置含有很多环境变量,我采取的方案是结合 env 文件和脚本进行统一替换,编写好配置文件后,依次将上述配置文件应用到部署集群后就可以正式对外提供服务了。完结撒花,如果有什么写的不对的地方,欢迎各位观众老爷指出。
短地址怎么来
这里有个小 PHP Demo,仅供参考。
function generate_unique_short_keys(string $string, int $short_len = 6) : array{$base_chars = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k','l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v','w', 'x', 'y', 'z', 'A', 'B', 'C', 'D', 'E', 'F', 'G','H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R','S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', '0', '1', '2','3', '4', '5', '6', '7', '8', '9'];$short_arr = [];$md5_arr = str_split(md5($string), 8);foreach ($md5_arr as $md5_str) {$hex = hexdec($md5_str) & 0x3fffffff;$short_str = null;for ($i = 0; $i < $short_len; $i++) {$index = 0x0000003d & $hex;$short_str .= $base_chars[$index];$hex = $hex >> intval(30 / $short_len);}array_push($short_arr, $short_str);}return $short_arr;}
