在微信小程序中使用protobuf（静态方式）

vlambda
2022-03-13

在微信小程序中使用protobuf（静态方式）

今天在开发中想结合最近的对于Go的学习，于是便想尝试将小程序后端请求中的JSON数据换成protobuf，这在其他语言甚至原生的js中都挺好实现的，有官方的语言转换工具就不用说了，但是小程序的框架比较特殊，基本上是屏蔽了一些动态执行以及反射相关的能力，这让protobuf以反射作为生成对象的核心实现无法完成。查找了许多方案，小程序虽然支持npm，但没有针对小程序进行优化的基本上都用不了，别说官方库了，即使是第三方的也无法使用，后来经过反复实现，总结出了一个方案，以后可能有更好的方案，这个方案是用静态生成的库和接口描述实现来使用protobuf

准备

首先用到的库是为了能尽可能轻量实现的protobuf.js^[1]

首先在全局安装pbjs工具，注意是全局不需要安装到项目中

npm -g intall protobufjs

测试pbjs命令是否可用，无法使用就需要先配置环境变量，然后在命令行定位到存有.proto协议描述文件的位置，我随便写了一个用于测试

//Logger.proto
syntax = "proto3";
package pb;

message loggerMsg {
  string logger_name = 1;
  bytes payload = 2;
}

然后运行pbjs，配置用于生成静态协议实现，其中static-module使用静态实现模式，-o指定输出文件，--es6用于生成ES6规范的js文件（项目采用ES6支持），-w commonjs很重要，将会把定义的协议直接静态写入，并只需最小支持库就可以运行，这对后续引入静态库很重要

pbjs -t static-module -o Logger.js --es6 -w commonjs .\Logger.proto

下载依赖库

接下来生成了Logger.js实现文件，然后根据官方文档，去下载最小实现库，体积很小，大概只有21K，如果你使用-t json来使用JSON描述文件，则需要light级别的库，而上文采用静态实现只需要minimal级别的最小库，具体说明可参考下图官方说明 https://github.com/protobufjs/protobuf.js/blob/master/cli/README.md#pbjs-for-javascript

For production environments it is recommended to bundle all your .proto files to a single .json file, which minimizes the number of network requests and avoids any parser overhead (hint: works with just the light library):

$> pbjs -t json file1.proto file2.proto > bundle.json

Now, either include this file in your final bundle:

var root = protobuf.Root.fromJSON(require("./bundle.json"));

or load it the usual way:

protobuf.load("bundle.json", function(err, root) {
    ...
});

Generated static code, on the other hand, works with just the minimal library. For example

$> pbjs -t static-module -w commonjs -o compiled.js file1.proto file2.proto

will generate static code for definitions within file1.proto and file2.proto to a CommonJS module compiled.js.

ProTip! Documenting your .proto files with /** ... */-blocks or (trailing) /// ... lines translates to generated static code.

下载可见 https://github.com/protobufjs/protobuf.js#distributions

Distributions

Where bundle size is a factor, there are additional stripped-down versions of the full library^[2] (~19kb gzipped) available that exclude certain functionality:

•When working with JSON descriptors (i.e. generated by pbjs^[3]) and/or reflection only, see the light library^[4] (~16kb gzipped) that excludes the parser. CommonJS entry point is:

var protobuf = require("protobufjs/light");

•When working with statically generated code only, see the minimal library^[5] (~6.5kb gzipped) that also excludes reflection. CommonJS entry point is:

var protobuf = require("protobufjs/minimal");

下载对应依赖后，要参考刚才生成的Logger.js，前几行就可以看到关于引入依赖的语句

var $protobuf = require("protobufjs/minimal");

为了未来生成的文件不用再修改，故需要将下载的最小库命名为minimal.js并放置在同目录的protobufjs文件夹下

使用

现在，这个协议实现应该就能使用了，导入这个静态文件的pb变量开始使用

import {pb} from './Logger'

先生成定义的消息对象loggerMsg

let log = pb.loggerMsg.create({
    logger_name: "logger",
    payload: new Uint8Array(10)
  })

payload字段属性为bytes，在js中常用Uint8Array，其它映射表如下 https://github.com/protobufjs/protobuf.js#valid-message

A valid message is an object (1) not missing any required fields and (2) exclusively composed of JS types understood by the wire format writer. There are two possible types of valid messages and the encoder is able to work with both of these for convenience:

•Message instances (explicit instances of message classes with default values on their prototype) always (have to) satisfy the requirements of a valid message by design and•Plain JavaScript objects that just so happen to be composed in a way satisfying the requirements of a valid message as well. In a nutshell, the wire format writer understands the following types:

Field type Expected JS type (create, encode) Conversion (fromObject)

s-/u-/int32
s-/fixed32 number (32 bit integer) value | 0 if signed
value >>> 0 if unsigned

s-/u-/int64
s-/fixed64 Long-like (optimal)
number (53 bit integer) Long.fromValue(value) with long.js
parseInt(value, 10) otherwise

float
double number Number(value)

bool boolean Boolean(value)

string string String(value)

bytes Uint8Array (optimal)
Buffer (optimal under node)
Array.<number> (8 bit integers) base64.decode(value) if a string
Object with non-zero .length is assumed to be buffer-like

enum number (32 bit integer) Looks up the numeric id if a string

message Valid message Message.fromObject(value)

Field type	Expected JS type (create, encode)	Conversion (fromObject)
s-/u-/int32 s-/fixed32	`number` (32 bit integer)	`value \| 0` if signed `value >>> 0` if unsigned
s-/u-/int64 s-/fixed64	`Long`-like (optimal) `number` (53 bit integer)	`Long.fromValue(value)` with long.js `parseInt(value, 10)` otherwise
float double	`number`	`Number(value)`
bool	`boolean`	`Boolean(value)`
string	`string`	`String(value)`
bytes	`Uint8Array` (optimal) `Buffer` (optimal under node) `Array.<number>` (8 bit integers)	`base64.decode(value)` if a `string` `Object` with non-zero `.length` is assumed to be buffer-like
enum	`number` (32 bit integer)	Looks up the numeric id if a `string`
message	Valid message	`Message.fromObject(value)`

•Explicit undefined and null are considered as not set if the field is optional.

•Repeated fields are Array.<T>.•Map fields are Object.<string,T> with the key being the string representation of the respective value or an 8 characters long binary hash string for Long-likes.•Types marked as optimal provide the best performance because no conversion step (i.e. number to low and high bits or base64 string to buffer) is required.

最后，利用这个对象完成编码

let buffer = pb.loggerMsg.encode(log).finish();
let decoded = pb.loggerMsg.decode(buffer);

自此，便实现了protobuf的编码与解码

References

[1] protobuf.js: https://github.com/protobufjs/protobuf.js
[2] full library: https://github.com/dcodeIO/protobuf.js/tree/master/dist
[3] pbjs: cli/README.md#pbjs-for-javascript
[4] light library: https://github.com/dcodeIO/protobuf.js/tree/master/dist/light
[5] minimal library: https://github.com/dcodeIO/protobuf.js/tree/master/dist/minimal

原文链接：https://blog.focot.cn/217