Binary encoding

For data that is used only internally within your organization, there is less pressure to use a lowest-common-denominator encoding format. For example, you could choose a format that is more compact or faster to parse. For a small dataset, the gains are negligible, but once you get into the terabytes, the choice of data format can have a big impact.

对于仅在组织内部使用的数据，使用最低公共编码格式的压力较小。例如，你可以选择更紧凑或更快速解析的格式。对于小数据集，收益微不足道，但是当你进入了以太字节时，数据格式的选择就会有很大的影响。

JSON is less verbose than XML, but both still use a lot of space compared to binary formats. This observation led to the development of a profusion of binary encodings for JSON (MessagePack, BSON, BJSON, UBJSON, BISON, and Smile, to name a few) and for XML (WBXML and Fast Infoset, for example). These formats have been adopted in various niches, but none of them are as widely adopted as the textual versions of JSON and XML.

JSON比XML更简洁，但与二进制格式相比，两者仍然使用很多空间。这个观察结果导致了大量针对JSON（如MessagePack、BSON、BJSON、UBJSON、BISON和Smile）和XML（例如WBXML和Fast Infoset）的二进制编码的开发。这些格式已被用于各种领域，但它们中没有一个像JSON和XML的文本版本那样被广泛采用。

Some of these formats extend the set of datatypes (e.g., distinguishing integers and floating-point numbers, or adding support for binary strings), but otherwise they keep the JSON/XML data model unchanged. In particular, since they don’t prescribe a schema, they need to include all the object field names within the encoded data. That is, in a binary encoding of the JSON document in Example 4-1 , they will need to include the strings userName , favoriteNumber , and interests somewhere.

其中一些格式扩展了数据类型集合（例如，区分整数和浮点数，或添加对二进制字符串的支持），但是它们保持了JSON/XML数据模型不变。特别地，由于它们不规定模式，因此它们需要在编码数据中包含所有对象字段名称。也就是说，在JSON文档的二进制编码中，它们需要在某个地方包含字符串userName、favoriteNumber和interests。

{
    "userName": "Martin",
    "favoriteNumber": 1337,
    "interests": ["daydreaming", "hacking"]
}

Let’s look at an example of MessagePack, a binary encoding for JSON. Figure 4-1 shows the byte sequence that you get if you encode the JSON document in Example 4-1 with MessagePack [ 14 ]. The first few bytes are as follows:

让我们来看一个例子，MessagePack是一种用于JSON的二进制编码。图4-1显示了在MessagePack [14]中对Example 4-1中的JSON文档进行编码时所获得的字节序列。前几个字节如下：

1. The first byte, 0x83 , indicates that what follows is an object (top four bits = 0x80 ) with three fields (bottom four bits = 0x03 ). (In case you’re wondering what happens if an object has more than 15 fields, so that the number of fields doesn’t fit in four bits, it then gets a different type indicator, and the number of fields is encoded in two or four bytes.)

   第一个字节0x83表示接下来的内容是一个对象（高四位=0x80），有三个域（低四位=0x03）。如果你想知道对象有超过15个属性的情况下，没有办法用四位来编码属性的数量，那么就用不同的类型指示符，并用两个或四个字节来编码属性数量。

2. The second byte, 0xa8 , indicates that what follows is a string (top four bits = 0xa0 ) that is eight bytes long (bottom four bits = 0x08 ).

   第二个字节0xa8表示接下来的内容是一个字符串（前四位=0xa0），并且该字符串共有8个字节长度（后四位=0x08）。

3. The next eight bytes are the field name userName in ASCII. Since the length was indicated previously, there’s no need for any marker to tell us where the string ends (or any escaping).

   下一个八个字节是ASCII码格式的字段名userName。由于长度已经先前标示，因此不需要任何标记来告诉我们字符串的结尾（或任何转义）。

4. The next seven bytes encode the six-letter string value Martin with a prefix 0xa6 , and so on.

   下七个字节编码了带有0xa6前缀的六个字母字符串值Martin，以此类推。

The binary encoding is 66 bytes long, which is only a little less than the 81 bytes taken by the textual JSON encoding (with whitespace removed). All the binary encodings of JSON are similar in this regard. It’s not clear whether such a small space reduction (and perhaps a speedup in parsing) is worth the loss of human-readability.

二进制编码长66字节，比文本JSON编码（无空格）占用的81字节略少。所有JSON的二进制编码在这方面都相似。尚不清楚这样的空间减少（可能会提高解析速度）是否值得失去人类可读性。

In the following sections we will see how we can do much better, and encode the same record in just 32 bytes.

在接下来的章节中，我们将看到我们如何可以做得更好，仅使用32个字节来编码相同的记录。

Figure 4-1. Example record ( Example 4-1 ) encoded using MessagePack.

Field tags and schema evolution

We said previously that schemas inevitably need to change over time. We call this schema evolution . How do Thrift and Protocol Buffers handle schema changes while keeping backward and forward compatibility?

我们之前曾经提到过，模式必然会随着时间的推移而发生改变。我们称之为模式演变。Thrift和协议缓冲如何在保持向前和向后兼容性的同时处理模式更改？

As you can see from the examples, an encoded record is just the concatenation of its encoded fields. Each field is identified by its tag number (the numbers 1 , 2 , 3 in the sample schemas) and annotated with a datatype (e.g., string or integer). If a field value is not set, it is simply omitted from the encoded record. From this you can see that field tags are critical to the meaning of the encoded data. You can change the name of a field in the schema, since the encoded data never refers to field names, but you cannot change a field’s tag, since that would make all existing encoded data invalid.

从这些示例中可以看出，编码记录只是其编码字段的串联。每个字段由其标签号（示例模式中的数字1、2、3）标识，并带有数据类型（例如，字符串或整数）的注释。如果字段值未设置，则会从编码记录中省略它。由此可以看出，字段标签对于编码数据的含义至关重要。你可以更改模式中字段的名称，因为编码数据从不引用字段名称，但你无法更改字段的标签，因为这将使所有现有的编码数据无效。

You can add new fields to the schema, provided that you give each field a new tag number. If old code (which doesn’t know about the new tag numbers you added) tries to read data written by new code, including a new field with a tag number it doesn’t recognize, it can simply ignore that field. The datatype annotation allows the parser to determine how many bytes it needs to skip. This maintains forward compatibility: old code can read records that were written by new code.

如果您给每个字段分配新标签号，那么可以向模式添加新字段。如果旧代码（不知道您添加的新标签号）尝试读取由新代码编写的数据，包括使用无法识别的标签号的新字段，则可以简单地忽略该字段。数据类型注释允许解析器确定需要跳过多少字节。 这保持向前兼容性：旧代码可以读取由新代码编写的记录。

What about backward compatibility? As long as each field has a unique tag number, new code can always read old data, because the tag numbers still have the same meaning. The only detail is that if you add a new field, you cannot make it required. If you were to add a field and make it required, that check would fail if new code read data written by old code, because the old code will not have written the new field that you added. Therefore, to maintain backward compatibility, every field you add after the initial deployment of the schema must be optional or have a default value.

对于向后兼容性的处理，只要每个字段都有唯一的标签号码，新代码就能够读取旧数据，因为标签号码的含义没有改变。唯一的问题是，如果你添加了一个新字段，就不能将其设为必填字段。如果你添加了一个必填字段，新代码读取旧代码写入的数据时，该检查将失败，因为旧代码将无法写入你添加的新字段。因此，为了保持向后兼容性，在架构的初始部署之后添加的每个字段都必须是可选的或者有一个默认值。

Removing a field is just like adding a field, with backward and forward compatibility concerns reversed. That means you can only remove a field that is optional (a required field can never be removed), and you can never use the same tag number again (because you may still have data written somewhere that includes the old tag number, and that field must be ignored by new code).

删除一个字段就像添加一个字段一样，需要考虑向前和向后兼容性问题的反转。这意味着你只能删除一个可选的字段（必需字段永远不能被删除），而且你永远不能再次使用相同的标签号码（因为你可能仍然有数据写在某个地方，包括旧的标签号码，而且新代码必须忽略该字段）。

Datatypes and schema evolution

What about changing the datatype of a field? That may be possible—check the documentation for details—but there is a risk that values will lose precision or get truncated. For example, say you change a 32-bit integer into a 64-bit integer. New code can easily read data written by old code, because the parser can fill in any missing bits with zeros. However, if old code reads data written by new code, the old code is still using a 32-bit variable to hold the value. If the decoded 64-bit value won’t fit in 32 bits, it will be truncated.

更改字段的数据类型怎么样？这可能是可能的-请检查文档以获取详细信息-但存在值将失去精度或被截断的风险。例如，假设您将32位整数更改为64位整数。新代码可以轻松读取旧代码编写的数据，因为解析器可以用零填充任何缺失的位。但是，如果旧代码读取新代码编写的数据，则旧代码仍在使用32位变量来保存该值。如果解码的64位值无法放入32位中，则将被截断。

A curious detail of Protocol Buffers is that it does not have a list or array datatype, but instead has a repeated marker for fields (which is a third option alongside required and optional ). As you can see in Figure 4-4 , the encoding of a repeated field is just what it says on the tin: the same field tag simply appears multiple times in the record. This has the nice effect that it’s okay to change an optional (single-valued) field into a repeated (multi-valued) field. New code reading old data sees a list with zero or one elements (depending on whether the field was present); old code reading new data sees only the last element of the list.

一个好奇的细节是，Protocol Buffers 没有列表或数组的数据类型，而是为字段添加了一个重复标记（这是 required 和 optional 之外的第三个选项）。如图 4-4 所示，重复字段的编码就像它的名字一样：同样的字段标签在记录中出现多次。这具有良好的效果，即可以将可选（单值）字段更改为重复的（多值）字段。读取旧数据的新代码会看到一个具有零个或一个元素的列表（取决于字段是否存在）；读取新数据的旧代码只会看到列表的最后一个元素。

Thrift has a dedicated list datatype, which is parameterized with the datatype of the list elements. This does not allow the same evolution from single-valued to multi-valued as Protocol Buffers does, but it has the advantage of supporting nested lists.

Thrift有一个专用的列表数据类型，它带有列表元素的数据类型参数。与Protocol Buffers不同，它不允许从单值到多值的相同演变，但它支持嵌套列表的优点。

The writer’s schema and the reader’s schema

With Avro, when an application wants to encode some data (to write it to a file or database, to send it over the network, etc.), it encodes the data using whatever version of the schema it knows about—for example, that schema may be compiled into the application. This is known as the writer’s schema .

当一个应用程序想要对一些数据进行编码（写入文件或数据库，通过网络发送等），使用Avro，它会使用任何它知道的模式版本来对数据进行编码 - 例如，该模式可能已编译到应用程序中。这被称为编写者的模式。

When an application wants to decode some data (read it from a file or database, receive it from the network, etc.), it is expecting the data to be in some schema, which is known as the reader’s schema . That is the schema the application code is relying on—code may have been generated from that schema during the application’s build process.

当一个应用程序想要解码一些数据（从文件或数据库中读取，从网络中接收等等），它期望数据以某种模式存在，这被称为读取者模式。这是应用程序代码所依赖的模式，代码可能在应用程序构建过程中从该模式中生成。

The key idea with Avro is that the writer’s schema and the reader’s schema don’t have to be the same —they only need to be compatible. When data is decoded (read), the Avro library resolves the differences by looking at the writer’s schema and the reader’s schema side by side and translating the data from the writer’s schema into the reader’s schema. The Avro specification [ 20 ] defines exactly how this resolution works, and it is illustrated in Figure 4-6 .

Avro的关键思想是，编写者的模式和读者的模式不必相同-它们只需要兼容。当数据被解码（读取）时，Avro库通过并排查看编写者的模式和读者的模式并将数据从编写者的模式转换为读者的模式来解决差异。 Avro规范明确定义了这种解析方式，并在图4-6中进行了说明。

For example, it’s no problem if the writer’s schema and the reader’s schema have their fields in a different order, because the schema resolution matches up the fields by field name. If the code reading the data encounters a field that appears in the writer’s schema but not in the reader’s schema, it is ignored. If the code reading the data expects some field, but the writer’s schema does not contain a field of that name, it is filled in with a default value declared in the reader’s schema.

例如，如果写入者的模式和读者的模式具有不同顺序的字段，也不会有问题，因为模式分辨率通过字段名称将字段匹配起来。如果读取数据的代码遇到出现在写入者模式中但不在读者模式中的字段，则会被忽略。如果读取数据的代码期望某个字段，但写入者的模式中不包含该名称的字段，则会使用读者模式中声明的默认值填充。

Figure 4-6. An Avro reader resolves differences between the writer’s schema and the reader’s schema.

Schema evolution rules

With Avro, forward compatibility means that you can have a new version of the schema as writer and an old version of the schema as reader. Conversely, backward compatibility means that you can have a new version of the schema as reader and an old version as writer.

使用 Avro，向前兼容性意味着可以将新版本的模式作为写入者，并将旧版本的模式作为读取者。反之，向后兼容性意味着可以将新版本的模式作为读取者，并将旧版本的模式作为写入者。

To maintain compatibility, you may only add or remove a field that has a default value. (The field favoriteNumber in our Avro schema has a default value of null .) For example, say you add a field with a default value, so this new field exists in the new schema but not the old one. When a reader using the new schema reads a record written with the old schema, the default value is filled in for the missing field.

为了保持兼容性，您只能添加或删除具有默认值的字段。（在我们的Avro模式中，favoriteNumber字段具有null的默认值）。例如，假设您添加了一个具有默认值的字段，因此此新字段存在于新模式中但不存在于旧模式中。当使用新模式的读取器读取使用旧模式编写的记录时，缺失字段的默认值将被填充。

If you were to add a field that has no default value, new readers wouldn’t be able to read data written by old writers, so you would break backward compatibility. If you were to remove a field that has no default value, old readers wouldn’t be able to read data written by new writers, so you would break forward compatibility.

如果您添加的字段没有默认值，新的读者将无法读取旧的写入的数据，这将破坏向后兼容性。如果您删除没有默认值的字段，旧的读者将无法读取新的写入的数据，这将破坏向前兼容性。

In some programming languages, null is an acceptable default for any variable, but this is not the case in Avro: if you want to allow a field to be null, you have to use a union type . For example, union { null, long, string } field; indicates that field can be a number, or a string, or null. You can only use null as a default value if it is one of the branches of the union. ^iv This is a little more verbose than having everything nullable by default, but it helps prevent bugs by being explicit about what can and cannot be null [ 22 ].

在一些编程语言中，null 是任何变量的可接受默认值，但在 Avro 中不是这种情况：如果您想要允许某个字段为空，则必须使用联合类型。例如，union {null，long，string} field; 表示字段可以是数字、字符串或 null。只有当 null 是联合类型的一个分支时，才能将其用作默认值。这比默认情况下所有内容都可为空要冗长一些，但通过明确规定哪些内容可以为空以及哪些内容不可以为空可以帮助预防出现错误 [22]。

Consequently, Avro doesn’t have optional and required markers in the same way as Protocol Buffers and Thrift do (it has union types and default values instead).

因此，Avro没有像Protocol Buffers和Thrift那样的可选和必需标记（它有联合类型和默认值）。

Changing the datatype of a field is possible, provided that Avro can convert the type. Changing the name of a field is possible but a little tricky: the reader’s schema can contain aliases for field names, so it can match an old writer’s schema field names against the aliases. This means that changing a field name is backward compatible but not forward compatible. Similarly, adding a branch to a union type is backward compatible but not forward compatible.

改变字段的数据类型是可能的，前提是Avro可以转换类型。改变字段名称是可能的，但有点棘手：读取器模式可以包含字段名称的别名，因此它可以将旧写入器模式字段名称与别名进行匹配。这意味着更改字段名称是向后兼容但不是向前兼容。同样，向联合类型添加分支是向后兼容但不是向前兼容。

But what is the writer’s schema?

There is an important question that we’ve glossed over so far: how does the reader know the writer’s schema with which a particular piece of data was encoded? We can’t just include the entire schema with every record, because the schema would likely be much bigger than the encoded data, making all the space savings from the binary encoding futile.

这里有一个重要的问题我们之前忽略了：读者如何知道写手用哪种模式来编码特定的数据？我们不能每个记录都包含整个模式，因为模式很可能比编码出来的数据还要大，这样二进制编码所节省的空间就变得无意义了。

The answer depends on the context in which Avro is being used. To give a few examples:

答案取决于Avro使用的上下文。举几个例子：

Large file with lots of records

A common use for Avro—especially in the context of Hadoop—is for storing a large file containing millions of records, all encoded with the same schema. (We will discuss this kind of situation in Chapter 10 .) In this case, the writer of that file can just include the writer’s schema once at the beginning of the file. Avro specifies a file format (object container files) to do this.

Avro的一种常见用途，尤其是在Hadoop的背景下，是用于存储包含数百万条记录、所有记录都使用相同模式进行编码的大型文件。（我们将在第10章中讨论这种情况。）在这种情况下，该文件的作者只需在文件开头包含一次作者的模式即可。Avro指定了一个文件格式（对象容器文件），用于执行此操作。

Database with individually written records

In a database, different records may be written at different points in time using different writer’s schemas—you cannot assume that all the records will have the same schema. The simplest solution is to include a version number at the beginning of every encoded record, and to keep a list of schema versions in your database. A reader can fetch a record, extract the version number, and then fetch the writer’s schema for that version number from the database. Using that writer’s schema, it can decode the rest of the record. (Espresso [ 23 ] works this way, for example.)

在数据库中，不同的记录可能使用不同的编写者模式在不同的时间点编写 - 您不能假设所有记录都具有相同的模式。最简单的解决方案是在每个编码记录的开头包含一个版本号，并在数据库中保留模式版本列表。读取器可以获取记录，提取版本号，然后从数据库中获取该版本号的编写者模式。使用该编写者模式，它可以解码其余的记录。（例如 Espresso 以此方式工作。）

Sending records over a network connection

When two processes are communicating over a bidirectional network connection, they can negotiate the schema version on connection setup and then use that schema for the lifetime of the connection. The Avro RPC protocol (see “Dataflow Through Services: REST and RPC” ) works like this.

当两个进程通过双向网络连接通信时，它们可以在连接设置期间协商模式版本，然后在连接的整个生命周期内使用该模式。Avro RPC协议（请参见“服务的数据流：REST和RPC”）是这样工作的。

A database of schema versions is a useful thing to have in any case, since it acts as documentation and gives you a chance to check schema compatibility [ 24 ]. As the version number, you could use a simple incrementing integer, or you could use a hash of the schema.

数据库模式版本的信息是任何情况下都有用的，因为它可以充当文档，同时还可以让你检查模式的兼容性[24]。你可以使用一个简单的递增整数作为版本号，或者你可以使用模式的哈希值。

Dynamically generated schemas

One advantage of Avro’s approach, compared to Protocol Buffers and Thrift, is that the schema doesn’t contain any tag numbers. But why is this important? What’s the problem with keeping a couple of numbers in the schema?

Avro的做法相对于Protocol Buffers和Thrift的优势之一就是模式中没有任何标签号。但这为什么重要呢？把几个数字写进模式有什么问题呢？ Avro的优势在于它不需要使用标签号进行编码，这样就可以避免出现不必要的翻译错误。

The difference is that Avro is friendlier to dynamically generated schemas. For example, say you have a relational database whose contents you want to dump to a file, and you want to use a binary format to avoid the aforementioned problems with textual formats (JSON, CSV, SQL). If you use Avro, you can fairly easily generate an Avro schema (in the JSON representation we saw earlier) from the relational schema and encode the database contents using that schema, dumping it all to an Avro object container file [ 25 ]. You generate a record schema for each database table, and each column becomes a field in that record. The column name in the database maps to the field name in Avro.

不同之处在于Avro更友好地支持动态生成模式。例如，假设您有一个关系型数据库，希望将其内容转储到文件中，并且想要使用二进制格式避免文本格式（JSON、CSV、SQL）的问题。如果使用Avro，您可以相对容易地从关系模式生成一个Avro模式（使用我们之前看到的JSON表示），并使用该模式对数据库内容进行编码，将其全部转储到Avro对象容器文件[25]中。您为每个数据库表生成一个记录模式，每个列成为该记录中的一个字段。数据库中的列名映射到Avro中的字段名。

Now, if the database schema changes (for example, a table has one column added and one column removed), you can just generate a new Avro schema from the updated database schema and export data in the new Avro schema. The data export process does not need to pay any attention to the schema change—it can simply do the schema conversion every time it runs. Anyone who reads the new data files will see that the fields of the record have changed, but since the fields are identified by name, the updated writer’s schema can still be matched up with the old reader’s schema.

现在，如果数据库模式发生变化（例如，添加了一列和删除了一列），您只需从更新的数据库模式生成一个新的Avro模式，并按新的Avro模式导出数据。数据导出过程不需要关注模式变化 - 它可以每次运行时进行模式转换。读取新数据文件的任何人都会看到记录的字段已更改，但由于字段是通过名称标识的，因此更新的写入器模式仍然可以与旧的读者模式匹配。

By contrast, if you were using Thrift or Protocol Buffers for this purpose, the field tags would likely have to be assigned by hand: every time the database schema changes, an administrator would have to manually update the mapping from database column names to field tags. (It might be possible to automate this, but the schema generator would have to be very careful to not assign previously used field tags.) This kind of dynamically generated schema simply wasn’t a design goal of Thrift or Protocol Buffers, whereas it was for Avro.

相比之下，如果您使用Thrift或Protocol Buffers来实现此目的，字段标签很可能需要手动分配：每当数据库模式更改时，管理员都必须手动更新从数据库列名称到字段标签的映射。（这可能是可以自动化的，但模式生成器必须非常小心，以不分配先前使用过的字段标签。）这种动态生成的模式根本不是Thrift或Protocol Buffers的设计目标，但却是Avro的设计目标。

Code generation and dynamically typed languages

Thrift and Protocol Buffers rely on code generation: after a schema has been defined, you can generate code that implements this schema in a programming language of your choice. This is useful in statically typed languages such as Java, C++, or C#, because it allows efficient in-memory structures to be used for decoded data, and it allows type checking and autocompletion in IDEs when writing programs that access the data structures.

节俭和协议缓冲区依赖代码生成：在定义模式之后，您可以生成实现此模式的代码，使用您选择的编程语言。这在静态类型语言（如Java、C++或C#）中非常有用，因为它允许使用高效的内存结构以解码数据，并在编写访问数据结构的程序时允许在IDE中进行类型检查和自动完成。

In dynamically typed programming languages such as JavaScript, Ruby, or Python, there is not much point in generating code, since there is no compile-time type checker to satisfy. Code generation is often frowned upon in these languages, since they otherwise avoid an explicit compilation step. Moreover, in the case of a dynamically generated schema (such as an Avro schema generated from a database table), code generation is an unnecessarily obstacle to getting to the data.

在动态类型的编程语言中，例如JavaScript，Ruby或Python，生成代码几乎没有意义，因为没有编译时类型检查器需要满足。在这些语言中，代码生成通常是不被赞成的，因为它们可以避免显式编译步骤。此外，在动态生成模式的情况下（例如从数据库表生成的Avro模式），代码生成是一个不必要的障碍，阻碍获取数据。

Avro provides optional code generation for statically typed programming languages, but it can be used just as well without any code generation. If you have an object container file (which embeds the writer’s schema), you can simply open it using the Avro library and look at the data in the same way as you could look at a JSON file. The file is self-describing since it includes all the necessary metadata.

Avro提供可选的代码生成，用于静态类型的编程语言，但是即使没有任何代码生成，它也可以很好地使用。如果您有一个对象容器文件（其中嵌入了作者模式），则可以使用Avro库打开它，并查看数据，就像您可以查看JSON文件一样。该文件是自描述的，因为它包含了所有必要的元数据。

This property is especially useful in conjunction with dynamically typed data processing languages like Apache Pig [ 26 ]. In Pig, you can just open some Avro files, start analyzing them, and write derived datasets to output files in Avro format without even thinking about schemas.

这个属性在动态类型的数据处理语言，比如Apache Pig[26]方面非常有用。在Pig中，你可以直接打开一些Avro文件，开始分析它们，并把得出的数据集以Avro格式写入输出文件，而无需考虑模式。

Different values written at different times

A database generally allows any value to be updated at any time. This means that within a single database you may have some values that were written five milliseconds ago, and some values that were written five years ago.

一个数据库通常允许任何值在任何时间进行更新。这意味着在单个数据库中，你可能会有一些值是五毫秒前写入的，也可能会有一些值是五年前写入的。

When you deploy a new version of your application (of a server-side application, at least), you may entirely replace the old version with the new version within a few minutes. The same is not true of database contents: the five-year-old data will still be there, in the original encoding, unless you have explicitly rewritten it since then. This observation is sometimes summed up as data outlives code .

当您部署新版本的应用程序（至少是服务器端应用程序）时，您可以在几分钟内完全使用新版本替换旧版本。但是，数据库内容则不同：5年前的数据仍然以原始编码存在，除非自那时以来您已明确重写它。这种观察有时被总结为数据超越代码。

Rewriting ( migrating ) data into a new schema is certainly possible, but it’s an expensive thing to do on a large dataset, so most databases avoid it if possible. Most relational databases allow simple schema changes, such as adding a new column with a null default value, without rewriting existing data. ^v When an old row is read, the database fills in null s for any columns that are missing from the encoded data on disk. LinkedIn’s document database Espresso uses Avro for storage, allowing it to use Avro’s schema evolution rules [ 23 ].

将数据重写（迁移）到新模式中肯定是可能的，但对于大型数据集而言是一项昂贵的工作，因此大多数数据库在可能的情况下避免这种情况。大多数关系型数据库允许简单的模式更改，例如添加一个默认值为 null 的新列，而无需重写现有数据。当读取旧行时，数据库会自动为任何缺失的列填充 null。 LinkedIn 的文档数据库 Espresso 使用 Avro 进行存储，从而可以使用 Avro 的模式演化规则[23]。

Schema evolution thus allows the entire database to appear as if it was encoded with a single schema, even though the underlying storage may contain records encoded with various historical versions of the schema.

模式演化因此允许整个数据库看起来像是使用单一模式编码，尽管底层存储可能包含使用各种历史版本模式编码的记录。

Archival storage

Perhaps you take a snapshot of your database from time to time, say for backup purposes or for loading into a data warehouse (see “Data Warehousing” ). In this case, the data dump will typically be encoded using the latest schema, even if the original encoding in the source database contained a mixture of schema versions from different eras. Since you’re copying the data anyway, you might as well encode the copy of the data consistently.

也许您会定期快照您的数据库，比如用于备份或加载到数据仓库（请参见“数据仓库”）。在这种情况下，数据转储通常将使用最新的模式进行编码，即使源数据库中的原始编码包含不同年代的模式版本的混合物。既然您已经在复制数据，那么也可以对数据副本进行一致的编码。

As the data dump is written in one go and is thereafter immutable, formats like Avro object container files are a good fit. This is also a good opportunity to encode the data in an analytics-friendly column-oriented format such as Parquet (see “Column Compression” ).

由于数据转储是一次性写入的，并且之后是不可变的，因此诸如Avro对象容器文件之类的格式非常适合。这也是将数据编码为适合分析的列定向格式（例如Parquet）的绝佳机会（请参见“列压缩”）。

In Chapter 10 we will talk more about using data in archival storage.

在第10章中，我们将更多地谈论如何在档案存储中使用数据。

Web services

When HTTP is used as the underlying protocol for talking to the service, it is called a web service . This is perhaps a slight misnomer, because web services are not only used on the web, but in several different contexts. For example:

当HTTP被用作与服务通信的基础协议时，就被称为web服务。这可能略微有误，因为web服务不仅在网络上使用，而且在几种不同的上下文中使用。例如：

1. A client application running on a user’s device (e.g., a native app on a mobile device, or JavaScript web app using Ajax) making requests to a service over HTTP. These requests typically go over the public internet.

   用户设备上运行的客户端应用程序（例如移动设备上的本机应用程序或使用Ajax的JavaScript Web应用程序），通过HTTP向服务发出请求。这些请求通常通过公共互联网进行。

2. One service making requests to another service owned by the same organization, often located within the same datacenter, as part of a service-oriented/microservices architecture. (Software that supports this kind of use case is sometimes called middleware .)

   一项服务向同一组织拥有的另一个服务发出请求，通常位于相同的数据中心，作为面向服务/微服务体系结构的一部分。（支持此类用例的软件有时称为中间件。）

3. One service making requests to a service owned by a different organization, usually via the internet. This is used for data exchange between different organizations’ backend systems. This category includes public APIs provided by online services, such as credit card processing systems, or OAuth for shared access to user data.

   一种服务向不同组织拥有的服务发出请求，通常通过互联网进行。这用于不同组织后端系统之间的数据交换。这个类别包括在线服务提供的公共API，例如信用卡处理系统，或用于共享访问用户数据的OAuth。

There are two popular approaches to web services: REST and SOAP . They are almost diametrically opposed in terms of philosophy, and often the subject of heated debate among their respective proponents. ^vi

有两种流行的 Web 服务方法：REST 和 SOAP。它们在哲学上几乎是完全相反的，常常是各自支持者之间的激烈辩论的主题。

REST is not a protocol, but rather a design philosophy that builds upon the principles of HTTP [ 34 , 35 ]. It emphasizes simple data formats, using URLs for identifying resources and using HTTP features for cache control, authentication, and content type negotiation. REST has been gaining popularity compared to SOAP, at least in the context of cross-organizational service integration [ 36 ], and is often associated with microservices [ 31 ]. An API designed according to the principles of REST is called RESTful .

REST不是一种协议，而是一种基于HTTP原则的设计哲学。 它强调使用简单的数据格式，使用URL来识别资源，并利用HTTP的缓存控制、认证和内容类型协商等特性。与SOAP相比，REST在跨组织服务集成的上下文中越来越受欢迎，并经常与微服务相关联。 根据REST原则设计的API称为RESTful。

By contrast, SOAP is an XML-based protocol for making network API requests. ^vii Although it is most commonly used over HTTP, it aims to be independent from HTTP and avoids using most HTTP features. Instead, it comes with a sprawling and complex multitude of related standards (the web service framework , known as WS-* ) that add various features [ 37 ].

相比之下，SOAP是一种基于XML的协议，用于进行网络API请求。尽管它最常用于HTTP上，但它旨在独立于HTTP，并避免使用大多数HTTP功能。相反，它带有庞大而复杂的相关标准（称为WS-*的Web服务框架），添加了各种功能。

The API of a SOAP web service is described using an XML-based language called the Web Services Description Language, or WSDL. WSDL enables code generation so that a client can access a remote service using local classes and method calls (which are encoded to XML messages and decoded again by the framework). This is useful in statically typed programming languages, but less so in dynamically typed ones (see “Code generation and dynamically typed languages” ).

SOAP网络服务的API是使用名为Web Services Description Language或WSDL的基于XML的语言来描述的。 WSDL使代码生成成为可能，以便客户端可以使用本地类和方法调用访问远程 服务（这些调用被编码为XML消息，并由框架进行解码）。 这对于静态类型的编程语言很有用，但在动态 类型的编程语言中不太有用（请参见“代码生成和动态类型的语言”）。

As WSDL is not designed to be human-readable, and as SOAP messages are often too complex to construct manually, users of SOAP rely heavily on tool support, code generation, and IDEs [ 38 ]. For users of programming languages that are not supported by SOAP vendors, integration with SOAP services is difficult.

由于 WSDL 不是以人类可读的方式设计的，而且 SOAP 消息经常太复杂而无法手工构造，因此使用 SOAP 的用户严重依赖工具支持、代码生成和 IDE [38]。对于不受 SOAP 供应商支持的编程语言用户来说，与 SOAP 服务集成是困难的。

Even though SOAP and its various extensions are ostensibly standardized, interoperability between different vendors’ implementations often causes problems [ 39 ]. For all of these reasons, although SOAP is still used in many large enterprises, it has fallen out of favor in most smaller companies.

尽管SOAP及其各种扩展显然是标准化的，但不同厂商实现间的互操作性经常会引起问题[39]。出于所有这些原因，虽然SOAP在许多大型企业中仍在使用，但在大多数较小的公司中已不再受欢迎。

RESTful APIs tend to favor simpler approaches, typically involving less code generation and automated tooling. A definition format such as OpenAPI, also known as Swagger [ 40 ], can be used to describe RESTful APIs and produce documentation.

RESTful APIs倾向于采用简单的方法，通常涉及较少的代码生成和自动化工具。定义格式，例如OpenAPI，也称为Swagger [40]，可用于描述RESTful APIs并生成文档。

The problems with remote procedure calls (RPCs)

Web services are merely the latest incarnation of a long line of technologies for making API requests over a network, many of which received a lot of hype but have serious problems. Enterprise JavaBeans (EJB) and Java’s Remote Method Invocation (RMI) are limited to Java. The Distributed Component Object Model (DCOM) is limited to Microsoft platforms. The Common Object Request Broker Architecture (CORBA) is excessively complex, and does not provide backward or forward compatibility [ 41 ].

网络服务仅是一系列通过网络进行API请求的技术中最新的一种，其中许多技术受到过大量宣传，但存在严重问题。企业JavaBeans(EJB) 和Java的远程方法调用(RMI)仅限于Java。分布式组件对象模型(DCOM)仅限于微软平台。通用对象请求代理架构(CORBA)过于复杂，并且不提供向前或向后兼容性[41]。

All of these are based on the idea of a remote procedure call (RPC), which has been around since the 1970s [ 42 ]. The RPC model tries to make a request to a remote network service look the same as calling a function or method in your programming language, within the same process (this abstraction is called location transparency ). Although RPC seems convenient at first, the approach is fundamentally flawed [ 43 , 44 ]. A network request is very different from a local function call:

所有这些都是基于远程过程调用（RPC）的概念，自1970年代以来一直存在[42]。 RPC模型试图使对远程网络服务的请求看起来与在同一进程中调用函数或方法相同（这个抽象称为位置透明度）。尽管RPC乍一看似乎很方便，但这种方法在根本上存在缺陷[43,44]。 网络请求与本地函数调用非常不同：

• A local function call is predictable and either succeeds or fails, depending only on parameters that are under your control. A network request is unpredictable: the request or response may be lost due to a network problem, or the remote machine may be slow or unavailable, and such problems are entirely outside of your control. Network problems are common, so you have to anticipate them, for example by retrying a failed request.

  本地函数调用是可预测的，仅取决于您控制的参数的成功或失败。网络请求不可预测：请求或响应可能由于网络问题丢失，远程计算机可能缓慢或不可用，并且此类问题完全超出您的控制范围。网络问题很常见，因此您必须预先考虑它们，例如通过重试失败的请求。

• A local function call either returns a result, or throws an exception, or never returns (because it goes into an infinite loop or the process crashes). A network request has another possible outcome: it may return without a result, due to a timeout . In that case, you simply don’t know what happened: if you don’t get a response from the remote service, you have no way of knowing whether the request got through or not. (We discuss this issue in more detail in Chapter 8 .)

  本地函数调用可能会返回结果，也可能会抛出异常，或者永远不会返回（因为进入无限循环或进程崩溃）。网络请求还有另一种可能的结果：由于超时而返回而没有结果。在这种情况下，你只是不知道发生了什么：如果你没有从远程服务收到响应，你就没有办法知道请求是否已经通过了。 （我们将在第8章中更详细地讨论这个问题。）

• If you retry a failed network request, it could happen that the requests are actually getting through, and only the responses are getting lost. In that case, retrying will cause the action to be performed multiple times, unless you build a mechanism for deduplication ( idempotence ) into the protocol. Local function calls don’t have this problem. (We discuss idempotence in more detail in Chapter 11 .)

  如果重新尝试一个失败的网络请求，可能会发生请求实际上能够通过，只是响应丢失的情况。在这种情况下，重新尝试会导致动作被执行多次，除非您在协议中构建重复消除（幂等性）机制。本地函数调用不会有这个问题。（我们将在第11章中更详细地讨论幂等性。）

• Every time you call a local function, it normally takes about the same time to execute. A network request is much slower than a function call, and its latency is also wildly variable: at good times it may complete in less than a millisecond, but when the network is congested or the remote service is overloaded it may take many seconds to do exactly the same thing.

  每次调用本地函数，它通常需要大致相同的时间来执行。网络请求比函数调用慢得多，而且延迟也极其不稳定：在良好的情况下，它可能在不到一毫秒的时间内完成，但当网络拥塞或远程服务过载时，完成相同的操作可能需要数秒钟的时间。

• When you call a local function, you can efficiently pass it references (pointers) to objects in local memory. When you make a network request, all those parameters need to be encoded into a sequence of bytes that can be sent over the network. That’s okay if the parameters are primitives like numbers or strings, but quickly becomes problematic with larger objects.

  当您调用本地函数时，您可以有效地将本地内存中的对象的引用（指针）传递给它。当您发出网络请求时，所有这些参数都需要被编码为可以发送到网络的字节序列。如果参数是原始类型如数字或字符串，那是没问题的，但是如果是大型对象，这会很快变得棘手。

• The client and the service may be implemented in different programming languages, so the RPC framework must translate datatypes from one language into another. This can end up ugly, since not all languages have the same types—recall JavaScript’s problems with numbers greater than 2 ⁵³ , for example (see “JSON, XML, and Binary Variants” ). This problem doesn’t exist in a single process written in a single language.

  客户端和服务可能是用不同的编程语言实现的，因此RPC框架必须将数据类型从一种语言转换为另一种语言。这可能会非常麻烦，因为不是所有语言都拥有相同的类型--例如JavaScript对大于253的数字的问题（请参见“JSON，XML和二进制变体”）。在单个进程中编写单个语言时不存在这个问题。

All of these factors mean that there’s no point trying to make a remote service look too much like a local object in your programming language, because it’s a fundamentally different thing. Part of the appeal of REST is that it doesn’t try to hide the fact that it’s a network protocol (although this doesn’t seem to stop people from building RPC libraries on top of REST).

所有这些因素意味着，在编程语言中没有必要试图让远程服务看起来过于像本地对象，因为它是完全不同的东西。 REST 的吸引力之一在于它不试图隐藏它是一个网络协议的事实（尽管这似乎并不能阻止人们在 REST 之上构建 RPC 库）。

Current directions for RPC

Despite all these problems, RPC isn’t going away. Various RPC frameworks have been built on top of all the encodings mentioned in this chapter: for example, Thrift and Avro come with RPC support included, gRPC is an RPC implementation using Protocol Buffers, Finagle also uses Thrift, and Rest.li uses JSON over HTTP.

尽管存在这些问题，RPC并不会消失。各种RPC框架已经基于本章提到的所有编码方式构建：例如，Thrift和Avro包含了RPC支持，gRPC是使用Protocol Buffers实现的RPC实施，Finagle还使用Thrift，而Rest.li使用JSON over HTTP。

This new generation of RPC frameworks is more explicit about the fact that a remote request is different from a local function call. For example, Finagle and Rest.li use futures ( promises ) to encapsulate asynchronous actions that may fail. Futures also simplify situations where you need to make requests to multiple services in parallel, and combine their results [ 45 ]. gRPC supports streams , where a call consists of not just one request and one response, but a series of requests and responses over time [ 46 ].

这一新一代的RPC框架更明确地表明远程请求与本地函数调用是不同的。例如，Finagle和Rest.li使用futures（承诺）来封装可能失败的异步操作。Futures还简化了需要并行向多个服务发出请求并合并其结果的情况[45]。 gRPC支持流式传输，在这种情况下，一个调用不仅仅包括一个请求和一个响应，还包括随时间的一系列请求和响应[46]。

Some of these frameworks also provide service discovery —that is, allowing a client to find out at which IP address and port number it can find a particular service. We will return to this topic in “Request Routing” .

其中一些框架还提供服务发现功能，即允许客户端查找特定服务的 IP 地址和端口号。我们在“请求路由”中会再次探讨这个话题。

Custom RPC protocols with a binary encoding format can achieve better performance than something generic like JSON over REST. However, a RESTful API has other significant advantages: it is good for experimentation and debugging (you can simply make requests to it using a web browser or the command-line tool curl , without any code generation or software installation), it is supported by all mainstream programming languages and platforms, and there is a vast ecosystem of tools available (servers, caches, load balancers, proxies, firewalls, monitoring, debugging tools, testing tools, etc.).

采用二进制编码格式的自定义RPC协议可以比通用的JSON over REST实现更好的性能。然而，RESTful API有其他重要的优势：它适用于实验和调试（您可以使用Web浏览器或命令行工具curl轻松地对其进行请求，无需任何代码生成或软件安装），它被所有主流编程语言和平台支持，并且有大量的工具生态系统可用（服务器、缓存、负载平衡器、代理、防火墙、监控、调试工具、测试工具等等）。

For these reasons, REST seems to be the predominant style for public APIs. The main focus of RPC frameworks is on requests between services owned by the same organization, typically within the same datacenter.

因此，REST似乎是公共API的主要风格。 RPC框架的主要重点是同一组织拥有的服务之间的请求，通常位于相同的数据中心。

Data encoding and evolution for RPC

For evolvability, it is important that RPC clients and servers can be changed and deployed independently. Compared to data flowing through databases (as described in the last section), we can make a simplifying assumption in the case of dataflow through services: it is reasonable to assume that all the servers will be updated first, and all the clients second. Thus, you only need backward compatibility on requests, and forward compatibility on responses.

对于可进化性，RPC客户端和服务器可以独立更改和部署非常重要。与通过数据库流动的数据相比（如上一节所述），对于通过服务进行的数据流动，我们可以进行简化的假设：合理地假设所有服务器将首先更新，而所有客户端将其次更新。因此，您只需要在请求方面进行向后兼容性，在响应方面进行向前兼容性。

The backward and forward compatibility properties of an RPC scheme are inherited from whatever encoding it uses:

RPC方案的向后和向前兼容性属性是从其使用的编码继承来的。

• Thrift, gRPC (Protocol Buffers), and Avro RPC can be evolved according to the compatibility rules of the respective encoding format.

  Thrift、gRPC（Protocol Buffers）和Avro RPC都可以根据各自编码格式的兼容性规则进行升级。

• In SOAP, requests and responses are specified with XML schemas. These can be evolved, but there are some subtle pitfalls [ 47 ].

  在SOAP中，请求和响应是使用XML模式指定的。它们可以演变，但有一些微妙的陷阱[47]。

• RESTful APIs most commonly use JSON (without a formally specified schema) for responses, and JSON or URI-encoded/form-encoded request parameters for requests. Adding optional request parameters and adding new fields to response objects are usually considered changes that maintain compatibility.

  RESTful API 最常用 JSON（没有正式指定的模式）作为响应，JSON 或 URI 编码/表单编码请求参数作为请求。添加可选请求参数和向响应对象添加新字段通常被认为是保持兼容性的变化。

Service compatibility is made harder by the fact that RPC is often used for communication across organizational boundaries, so the provider of a service often has no control over its clients and cannot force them to upgrade. Thus, compatibility needs to be maintained for a long time, perhaps indefinitely. If a compatibility-breaking change is required, the service provider often ends up maintaining multiple versions of the service API side by side.

由于RPC通常用于跨组织边界的通信，因此服务兼容性变得更加困难，因此服务提供者通常无法控制其客户端并强制其升级。因此，兼容性需要长期维护，可能无限期地维护。如果需要破坏兼容性的更改，则服务提供者通常会同时维护多个版本的服务API。

There is no agreement on how API versioning should work (i.e., how a client can indicate which version of the API it wants to use [ 48 ]). For RESTful APIs, common approaches are to use a version number in the URL or in the HTTP Accept header. For services that use API keys to identify a particular client, another option is to store a client’s requested API version on the server and to allow this version selection to be updated through a separate administrative interface [ 49 ].

API版本问题没有明确的协议，也就是说客户端如何指定要使用API的哪个版本[48]. 对于RESTful API，常见的方法是在URL或HTTP Accept头中使用版本号。对于使用API密钥标识特定客户端的服务，另一个选择是在服务器上存储客户端请求的API版本，并允许通过单独的管理接口更新此版本选择[49].

Message brokers

In the past, the landscape of message brokers was dominated by commercial enterprise software from companies such as TIBCO, IBM WebSphere, and webMethods. More recently, open source implementations such as RabbitMQ, ActiveMQ, HornetQ, NATS, and Apache Kafka have become popular. We will compare them in more detail in Chapter 11 .

过去，消息代理的市场被TIBCO、IBM WebSphere和webMethods等公司的商业企业软件所主导。近年来，像RabbitMQ、ActiveMQ、HornetQ、NATS和Apache Kafka这样的开源实现变得流行起来。我们将在第11章中对它们进行更详细的比较。

The detailed delivery semantics vary by implementation and configuration, but in general, message brokers are used as follows: one process sends a message to a named queue or topic , and the broker ensures that the message is delivered to one or more consumers of or subscribers to that queue or topic. There can be many producers and many consumers on the same topic.

具体的传递语义因实现和配置而异，但通常情况下，消息代理如下使用：一个进程向命名队列或主题发送一条消息，代理确保该消息传递给一个或多个订阅该队列或主题的消费者。同一个主题上可以有多个生产者和多个消费者。

A topic provides only one-way dataflow. However, a consumer may itself publish messages to another topic (so you can chain them together, as we shall see in Chapter 11 ), or to a reply queue that is consumed by the sender of the original message (allowing a request/response dataflow, similar to RPC).

一个主题只提供单向数据流。然而，消费者本身可以发布消息到另一个主题（因此可以将它们链接在一起，如第11章所示），或者发布到由原始消息发送者消费的回复队列（允许请求/响应数据流，类似于RPC）。

Message brokers typically don’t enforce any particular data model—a message is just a sequence of bytes with some metadata, so you can use any encoding format. If the encoding is backward and forward compatible, you have the greatest flexibility to change publishers and consumers independently and deploy them in any order.

通常，消息代理不强制任何特定的数据模型-消息只是一系列字节和一些元数据，因此您可以使用任何编码格式。如果编码是向后和向前兼容的，则您可以最大限度地灵活地更改发布者和消费者的顺序，并将它们独立部署。

If a consumer republishes messages to another topic, you may need to be careful to preserve unknown fields, to prevent the issue described previously in the context of databases ( Figure 4-7 ).

如果消费者将消息重新发布到另一个主题，则需要小心保留未知字段，以防止在数据库上下文中描述的问题（图4-7）。

Distributed actor frameworks

The actor model is a programming model for concurrency in a single process. Rather than dealing directly with threads (and the associated problems of race conditions, locking, and deadlock), logic is encapsulated in actors . Each actor typically represents one client or entity, it may have some local state (which is not shared with any other actor), and it communicates with other actors by sending and receiving asynchronous messages. Message delivery is not guaranteed: in certain error scenarios, messages will be lost. Since each actor processes only one message at a time, it doesn’t need to worry about threads, and each actor can be scheduled independently by the framework.

演员模型是单个进程并发的编程模型。与直接处理线程（和相关的竞态条件、锁定和死锁问题）不同，逻辑被封装在演员中。每个演员通常代表一个客户或实体，它可能有一些本地状态（不与任何其他演员共享），并通过发送和接收异步消息与其他演员通信。消息传递不是保证的：在某些错误情况下，消息将被丢失。由于每个演员只处理一个消息，因此它不需要担心线程，并且每个演员可以由框架独立调度。

In distributed actor frameworks , this programming model is used to scale an application across multiple nodes. The same message-passing mechanism is used, no matter whether the sender and recipient are on the same node or different nodes. If they are on different nodes, the message is transparently encoded into a byte sequence, sent over the network, and decoded on the other side.

在分布式 actor 框架中，该编程模型被用于跨多个节点扩展应用程序。无论发送方和接收方是在同一节点还是不同节点，都使用相同的消息传递机制。如果它们在不同的节点上，消息将被透明地编码为字节序列，通过网络发送，并在另一侧进行解码。

Location transparency works better in the actor model than in RPC, because the actor model already assumes that messages may be lost, even within a single process. Although latency over the network is likely higher than within the same process, there is less of a fundamental mismatch between local and remote communication when using the actor model.

在Actor模型中，位置透明度比RPC更好，因为Actor模型已经假定消息可能会丢失，即使在单个进程中也是如此。虽然网络延迟可能比同一进程内的延迟要高，但使用Actor模型时，本地和远程通信之间的基本不匹配要少得多。

A distributed actor framework essentially integrates a message broker and the actor programming model into a single framework. However, if you want to perform rolling upgrades of your actor-based application, you still have to worry about forward and backward compatibility, as messages may be sent from a node running the new version to a node running the old version, and vice versa.

分布式演员框架基本上将消息代理和演员编程模型集成到单个框架中。然而，如果您想要执行基于演员的应用程序的滚动升级，则仍然需要考虑前向和后向兼容性，因为消息可能会从运行新版本的节点发送到运行旧版本的节点，反之亦然。

Three popular distributed actor frameworks handle message encoding as follows:

三个流行的分布式 actor 框架的信息编码方式如下：

• Akka uses Java’s built-in serialization by default, which does not provide forward or backward compatibility. However, you can replace it with something like Protocol Buffers, and thus gain the ability to do rolling upgrades [ 50 ].

  Akka默认使用Java内置的序列化，但是它没有提供向前或向后兼容性。不过，您可以使用类似Protocol Buffers的东西替换它，从而获得滚动升级的能力[50].

• Orleans by default uses a custom data encoding format that does not support rolling upgrade deployments; to deploy a new version of your application, you need to set up a new cluster, move traffic from the old cluster to the new one, and shut down the old one [ 51 , 52 ]. Like with Akka, custom serialization plug-ins can be used.

  Orleans 默认使用自定义数据编码格式，不支持滚动升级部署；要部署应用程序的新版本，需要设置一个新的集群，将流量从旧集群移动到新集群，并关闭旧集群。与 Akka 一样，可以使用自定义序列化插件。

• In Erlang OTP it is surprisingly hard to make changes to record schemas (despite the system having many features designed for high availability); rolling upgrades are possible but need to be planned carefully [ 53 ]. An experimental new maps datatype (a JSON-like structure, introduced in Erlang R17 in 2014) may make this easier in the future [ 54 ].

  在Erlang OTP中，尽管该系统有许多为高可用性而设计的功能，但更改记录模式仍然令人惊讶地困难; 滚动升级可能是可行的，但需要仔细计划。一个实验性的新映射数据类型（类似于JSON的结构，在2014年Erlang R17中引入）可能会在未来使这一过程更加容易。