Plugins in Go – Eli Bendersky’s website

Related Articles

A few years ago I started writing a series of posts on the subject Additives: How they are designed and implemented in different programming systems and languages. In it, I will expand the series by providing some examples of plugins in Go.

As you may recall, the original post in this series identifies four basic plugin concepts, and argues that almost all plugin systems can be characterized and understood by mapping their design to these concepts:

  1. discovery
  2. enrollment
  3. App hooks to which plugins are attached (also known as “mounting points”)
  4. Exposing application capabilities back to plug-ins (also known as extension API)

Two types of additives

As with other programming languages ​​with static compilation, in Go it is common to talk about two general types of extensions: extensions during compilation and runtime extensions. We will review both types here.

Additives while compiling

Extensions during compilation consist of code packets that go into the application’s main binary. Once the binary is built, its functionality is fixed.

The best known example of an add-on system while compiling in Go is drivers for
Database / sql package. I wrote an entire post on this topic – please read it!

In a nutshell: Database drivers are packages that the main application imports using Blank Import _ "Name". These packages are then used
init Functions sign up with Database / sql Package via

Write the basic plugin concepts, here’s how plugins while compiling (and
Database / sql As a concrete example) Tariff:

  1. Disclosure: Very explicit, with import of add-on package. Extensions can then automatically register them init function.
  2. Registration: Because the plugin is integrated into the main application, it is simple for it to activate a registration function (e.g. sql.Register) Directly from the plugin.
  3. Hooks of an application: Typically, a plugin implements an interface that the application provides and the registration process will connect the interface application. With Database / sql, Plugin implemented the
    Driver The interface and value that implements this interface will be registered using sql.Register.
  4. Exposing application capabilities back to plugins: with plugins while compiling it is simple; Because the plugin is complex into binary, it can be simple will come Service packs from the main app and use them in its code as needed.

Extensions while running

Runtime extensions consist of code not captured in the original application’s binary; Instead, it connects to this application at runtime. In compiled languages, a common tool for achieving this goal is Shared libraries, And Go also supports this approach. The rest of this section will provide an example of developing a plugin system in Go using shared directories; Alternative approaches will be explored a little later.

Go comes with a Essay A built-in package in the regular library. This package allows us to write Go programs that go into shared directories instead of executable binaries; In addition, it provides simple functions for loading shared directories and getting icons from them.

For this post, I have developed a complete example of a plugin system while running; It replicates the source htmlize An example from the post about plugin infrastructure, and its design is similar to the one in the next post about plugins in C. The example consists of a simplistic program for converting some markup language (like reStructuredText Or Markdown) to HTML, with plug-in support that allows us to customize the way certain markup elements are processed. The full sample code for this post Available here.

The contents of the plugin library directory

Let us examine this sample using the basic concepts of plugins.

Discovery and registration: Performed using a file system search. The main app has a Essay Package with the LoadPlugins function. This function scans a given directory for files ending in .so And treats all these files as plugins. He expects to find a global function called
InitPlugin In any common library, and calls on her to provide him a
PluginManager (More on that below).

How the plugins became .so Files in the first place? By building with -buildmode = plugin flag. Look at README script and file In the code example
for further details.

As an application: Now is a good time to describe the PluginManager
Type. This is the main type used for communication between plugins and the main app. The flow is as follows:

  • The application creates a new one PluginManager B LoadPlugins, And passes it to all the plugins it finds.
  • Each plugin uses PluginManager Register its own therapists for different hooks.
  • LoadPlugins Returns the PluginManager To the main app after all the plugins have been registered to it.
  • When the application is running, it is using PluginManager To enable Hawks sign up for the plugin as needed.

For example, PluginManager There is this method:

func (pm *PluginManager) RegisterRoleHook(rolename string, hook RoleHook)

where RoleHook Is function type:

// RoleHook takes the role contents, DB and Post and returns the text this role
// should be replaced with.
type RoleHook func(string, *content.DB, *content.Post) string

Plugins can run RegisterRoleHook To register a therapist for a specific text role. Note that this design does not use Go interfaces, but alternative designs can do so; It all depends on the specific details of the application.

Exposing the app’s capabilities back to pluginsA: As you can see in
RoleHook Type up, the app transfers data objects to the plugin to use. Content.DB Provides access to the application database, and
Content.Post Provides the specific post that the plugin is currently designing. The plugin can use these objects to obtain data or behavior from the app, as needed.

Problems with shared library extensions in Go

Runtime extensions that use shared libraries and Essay The package works well for Go, as demonstrated in the previous section. However, this approach also has some serious drawbacks. The most important downside is that Go is very picky in maintaining the compatibility of the main app and shared directories it loads.

As an experiment, make a small change in one of the packages used for both the plugins and the main application, rebuild the main application and run it. You’ll probably get this error:

"plugin was built with a different version of package XXX"

This is because Go wants all versions of all the packages in the main app and plugins to fit exactly. It is clear what drives this: safety.

Consider C and C ++ as a counter example. In these languages, an app can load a shared directory with Delufen And then use dlsym Obtain symbols from it. dlsym Typing is extremely weak; It takes a symbol name and returns a void*. It is up to the user to assign it to a specific function type. If the type of function changes due to a version update, it is likely that the result could be some sort of segmentation glitch or even memory corruption.

Because Go relies on shared directories from plugins, it has the same built-in security issues. It tries to protect programmers from shooting themselves in the foot by promising that the app was built with the same versions of packages as all of its plugins. This helps prevent mismatches. In addition, the version of the Go compiler used to build the app and plugins must fit exactly.

However, this protection has drawbacks – which makes plugin development somewhat cumbersome. The need to rebuild all plugins whenever common packages change – even in ways that do not affect the plugin interface – is a heavy burden. Especially considering that by their very nature plugins are usually developed separately from the main app; They may live in separate reservoirs, have separate release rates, and so on.

Alternative approaches to runtime extensions

Given that Essay The package was added only to Go in Version 1.8 And the limitation described earlier, it is not surprising that Go’s ecosystem has seen the emergence of alternative plugin approaches.

One of the most interesting directions, IMHO, includes plugins via RPC. I’ve always been a fan of disconnecting an app for separate processes that call via RPC or just TCP on localhost (I guess they call it Micro services Nowadays), because it has some important advantages:

  • Isolation: A crash in the plugin does not download the entire app.
  • Interoperability between languages: If RPC is the interface, do you care what language the plugin is written in?
  • Distribution: If plugins interface through the web, we can easily distribute them to run on different machines to improve performance, reliability, etc.

Furthermore, Go is particularly facilitated by a fairly high-capacity RPC package in the standard library: net / rpc.

One of the most common RPC-based plug-in systems is hashicorp / go-plugin. Hashicorp is known for creating great Go software, and they probably use it go-plugin For many of their systems, so he was tested in battle (though his documentation could be better!)

go-plugin Ran on top net / rpc But also supports gRPC. Advanced RPC protocols like gRPC are well-suited for plug-ins because they involve out-of-the-box version management, tackling the difficult operating problem between different versions of plugins compared to the main app.



Please enter your comment!
Please enter your name here

Popular Articles