Blog

Cross Language Object Parsing

I wrote code that writes code

BinHong Lee

August 9, 2020

I was working on a side project where I had the backend and frontend setup to run on different programming languages. As I was working on it, it started to bug me a little that I have to write the same object / struct and enum for each language in use everytime I introduce a new enum or struct. It can also be confusing / dangerous if I updated one side and forget to update the other.

At first, I just wrote some shell script to automate this but it quickly become unfeasible. (Clarification: it’s still possible to do this, but without any sort of object oriented support, the code gets quite messy quickly and I didn’t want to maintain that.) Now, at this point I should let you know that there are actually multiple existing solutions (Apache Thrift or Protobuf) that are way more matured, robust, and designed for use in a much larger scale. If you’re working on an enterprise level solution (or looking to learn something that’s more transferable into your future work), you might want to take look into them instead. I personally wanted something simpler and more flexible so I began working on wings.

Installation

If you have nimble (comes with nim) installed, you can install wings with nimble install wings. As of now, the only alternative to that is to get the binary and place it in a folder included by the path (like /usr/bin/) which unfortunately means that it won’t automatically update when a newer version is released. I’m still working on (and learning how to) deploy it to more package managers for easier installation process.

Usage

You write a wings file like the example below then run wings <FILENAME>.wings.

go-filepath examples/go/classroom
kt-filepath examples/kt
nim-filepath examples/nim
py-filepath examples/py
ts-filepath examples/ts

py-import examples.output.py.people
ts-import { IWingsStruct }:wings-ts-util
import examples/input/emotion.wings
import examples/input/homework.wings

py-implement People
ts-implement IWingsStruct

# Any person who is studying in a class

struct Student {
  id          int       -1
  name        str
  cur_class   str
  feeling     Emotion   Emotion.Meh
  is_active   bool
  year        date
  graduation  date
  homeworks   []Homework
  something   Map<str,str>
}

ts-func(
  public addHomework(hw: Homework): void {
    this.Homeworks.push(hw);
  }
)

As you can see, not only does it support generating files based on given specific parameters, it also supports importing other wings files directly which will also be generated by wings (unless specified otherwise in your config file).

Library

Since it’s very beginning, wings has been published on nimble as a library for nim. With this, you will be able to make further customizations by importing it as a library especially if you only want to use a specific part of it. From there, you can even use build tools to incorporate it as part of your workflow if supported. (Shameless plug, I wrote the PR for please’s basic nim and nimble support through pleasings. Though admittedly it isn’t quite as complete as you can see here I use my own version of the config in the repo.)

I’ve also created stones, a separate library that contains some generic functions and tools used in wings.

Generated Files

Struct

Unlike the afromentioned existing alternatives, it doesn’t change the way the data was stored and transferred. You’ll still be converting them from and to JSON and communicate through JSON (so they will still be human readable). Therefore, comparatively, it will be “slower” than the alternatives as it will inherit the same “slowness” of JSON (depending on how it is serialized and deserialized in each individual languages) and bulkiness (it’s a pretty well established fact that JSON string takes more space in memory than something like a serialized thrift object).

Enum

Nothing much about them. They’re just normal enums that starts from 0 and increment by 1 for each element in it. Custom type and initialization is in the works and might change up the syntax a little when support is added.

Util Classes

While the generated files (generally) work out of the box, they by themselves are still lacking of certain features that can be applied across all Struct or Enum classes. This is where the util classes comes in. They are written in each of their respective targeted languages as a package / library. Currently, there’s only a Util class for TypeScript since golang already have all the features in the util class baked in with its JSON support.

Config

While the use of a config file is optional, its also where the core value “customizable” comes from. Config files are accpeted in JSON format and should be added as an argument when calling wings with -c:<CONFIG.json>. Some of the more simple / straightforward configurable value includes things like logging which controls the verbosity of the output message in the terminal, skipImport which lets you dictate if you’d want to also generate imported files or skip doing so, and header which lets you specify your customized the header comment for all the generated files. On top of that, you can also configure how you want the output files to be formatted by specifying your own templates in langConfig (more on this below).

Template

As mentioned above, you can also write your own or import someone else’s templates (with remoteLangConfigs) to configure the output files in your own format or even to output to languages that are not officially supported. This allows wings to be used in a more dynamic way both in terms of range of supported languages and style format instead of being limited by what’s officially supported (like in Thrift and protobuf though they do already have a wide range of support).

Takeaways

Overall, I think one of the main takeaway is that if there’s already an actively maintained existing solution, it’s probably better to use that than to create your own. By now you might have realizes that this project is slowly morphing into something that looks similar to the existing alternatives but in a much simpler form.

Admittedly, I’ve only started learning about how a programming language is written only after a few months into the project so the design and structure is not ideal. However, since this is also much simpler compared to a full blown programming language, I can still get away with it on this. That said, I might start to incorporate some of these principles into wings as I continue to work on it (including writing a proper lexer and parser for it).

Of course, this is just a little side project spawn off of another side project. There’s also the long time dilemma of “should I just write them manually or do I need to write codes (or use an external library) to write them” (the NIH syndrome). In this case the later took way more time than the former, but this is a side project so it’s more about learning than meeting non-existent delivery dates anyway. I’ve also find it fun to have been working on the project so far and I hope you find this helpful. If you’d like to learn more about it (or use it yourself), check out the project website at wings.sh. If you like what you’ve read, I’d appreciate a star on the wings repository.