META Web Manifesto

What is META Web? How it works and what its motivations? Original manifesto from February 2017.

I can't remember when it first started. A few years ago when I started developing our company information system I begin to see many issues and redundant work in the way how web applications were being developed. And from that moment I was noticing these issues more and more frequently. About two years ago the idea of META Web was born. I was developing it privately until today when I decided to publish it to a broader audience.

Jiří Hýbek, February 2017

What is META Web?

META Web is a concept of web technology which is defines API protocols and UI markup language to change the way how web applications are being developed nowadays and to improve the development and also the communication between web services and user interfaces.

Issues of today

I see the following web applications development issues as critical.

Diverse and incomprehensible APIs

Various applications and web services communicate with each other using application interfaces - APIs. There are many different formats (JSON, XML) and various communication protocols such as REST or RPC. But when we want to integrate two applications we must study and comply with their VARIOUS protocols, understand how they work and how to implement them.

It's such a Babylonian problem. I'm sure that you have already heard the story about the Tower of Babel where god punished people by diversifying their language. APIs represent similar story. Applications need dictionaries (individual implementations) to be able to communicate with each other.

Each API has different ways to access data, various ways to call methods, various data structures, data types and so on. Subsequently, developers must write code for almost every single service they want to integrate.

We have some standards of course but they are too specific, such as OAuth.

Complicated and difficult to develop user interfaces

User interfaces represent the second main part of applications development. Nowadays, there are many different platforms such as Windows, Linux, OSX, Android or iOS. Every platform is specific and if we want to create applications for each of these platforms we have only a few options. We can use some framework or library which provides an option to write applications once and compile them for all platforms. And we also have the WEB. A technology which is available to all platforms.

Personally, I see the web as the best option to create multiplatform applications. But there are many issues as well.

The web platform is universal and easy to understand at first sight, but if you submerge in it you will see that it is a true mayhem. There are many languages such as HTML, CSS and JavaScript (with their dialects: TypeScript, CoffeeScript, LESS), and various frameworks such as React JS or Angular. That means that working with the web platform only is not simple and effective. If we want to simplify work with the web platform we need to use a lot of code.

Too much ado for a simple goal.

And what is our goal? Our goal is to create web applications which have many things in common - navigation, lists, images, text content, connection to data. So why do we need tens of languages, frameworks and libraries to achieve something that simple? The old web itself (I mean hypertext) was created for almost the same purpose. HTML supports content tags, images, forms, tables. The only issue is that these features in themselves (in the core of HTML) are insufficient. This is the reason why we need so much extra code.

But the current frameworks for the client side provide similar features as HTML - extended input validation, table filtering, displaying of live data, communication with server using API and preferably without reloading a page. Why cannot the web platform integrate these features into its codebase?

Finally, there is the issue of design. Many web applications look almost the same so why bother? They have the same components, a clear and simple appearance. This fact results in traditional web pages increasingly looking like desktop and mobile applications.

Another common issue of user interfaces (of both applications and web pages) is their diversity. People are creatures of habit and they don't like change. Historically, navigations were always placed on the left side of a web page. Nowadays, primary navigations are being placed at the top. When you present something people are not used to they have difficulty working with it. The elderly or the handicapped, for example, find it hard to recognize a button if it looks different from the native system component.

Application development complexity and the DRY principle issues

Many programmers know the DRY principle (Don’t Repeat Yourself) which says that we have to avoid repetition of the same parts of code to achieve more efficient work and clarity of code.

But what does the reality look like? Let's use a simple application example. Imagine an address book app. It is a collection of records - contacts. Each contact consists of several fields - first name, last name, address, e-mail, phone number and so on. There are the following parts of this application:

  • Database - a standard application has a database where records are stored and where the record structure is defined - contact fields.
  • Application server - next we have an application server which enables interaction between the database and the user interface. It adds some logic and most importantly it handles request validations - it validates contact fields. When a client or an attacker sends an invalid request to the server it must send an error response.
  • Web / mobile application - user interfaces which provide ability to interact with the application. It consists of a contact list, a contact detail, an add form, an edit form, etc. So in this part we have to specify all contact fields again.

What do these parts have in common? The definition of contact fields. In each application part we need to specify all contact fields in code. We have to define them in the database, in the application server code and also in the user interface. So, if we have 6 fields then we need to code them 18 times at 3 different application parts.

But their definitions are almost the same at all the parts. We specify the field's name, its type (text / number / e-mail), validation rules, if it is mandatory and if it's linked to other records and so on.

So why do we need to code this so many times? We actually don't. First we can create a shared record definition which enables us to create a database structure, validate requests and also display and validate user interface fields.

Although we try not to break the DRY principle, the current web technologies are forcing us to violate this principle.

Solution? The META Web

The goal of the META Web is to find a solution to the previously mentioned issues.

The META Web concept consists of:

  1. META API - the goal is to define united and general API for communication between web services, user interfaces and servers.
  2. META UI - the goal is to define universal markup language which will provide a tool to build user interfaces with simplicity and without the need for breaking the DRY principle and without the need to use various frameworks and redundant code.
  3. META Script - the goal is to create simple scripting language (something like table processor expressions) which will allow dynamic evaluation of data at META UI layer.
  4. META Dictionary - the goal is to define a common dictionary of standard objects, properties and types - may be based on schema.org.

META API

The goal of the generic META API protocol is to define a universal and common way how to describe web services and how to communicate between them thus providing an effective tool for integration and to avoid the Babylonian problem.

META API consists of:

  • Schema definition
  • Request / response formats

The schema definition describes service resources, its properties, capabilities and methods. Schema also consists of descriptive properties thus allowing other services, developers and user interfaces to understand the service data model.

We already have similar projects which address the same issue as Swagger, WSDL, Hydra or JSON-LD but it will be difficult to integrate them into the META Web concept.

META UI

The goal of META UI markup languages is to provide mechanisms for effective development of multiplatform user interfaces.

The META UI document defines the application view. For example a collection of contacts or contact detail. It also adds the navigation layer.

META UI defines:

  • Data source - where to get data and how to store them.
  • Visual components - how to present data to a user.

It's not anything unusual. But the effect comes with integration with META API. When we define the connection to META API data source we already have a description of the data model available. So we know which properties the resource has - with the contact app example we automatically know that our contact record has a first name, last name, we know how to label these properties, which type they are, how to validate them or how we can filter contacts collection and so on.

So in the META UI we don't need to define fields and columns anymore, which allows us to avoid breaking the DRY principle. Also when we change the data model on a server then these changes will be automatically reflected on the client-side application without any reprogramming.

The only thing we have to deal with is a UI layout. The META UI allows us to define composition of view components, which I think is everything we need on a the client side.

META Script and META Dictionary

I will describe these parts another day.

Examples of META API and META UI

I would like to describe the basic idea of using the META API and META UI for a better understanding of the concept.

This is just an example, the concept is under active development and is changing rapidly.

The following demo is based on the address book example.

Web service - using META API

Our web service has two endpoints which provide:

  1. Collection of contacts
  2. Individual contact record

META API is based on OOP principle. So every endpoint is an object with properties and methods.

  • Calling GET method will return object properties with values.
  • Calling POST method will invoke object method
  • Calling PUT method will set object property values
  • Calling DELETE method will delete resource if supported
  • Calling OPTIONS method will return resource schema (META API description)

But the methods described in the schema are different from HTTP words. They describe different resource capabilities:

  • Method GET means that resource properties can be retrieved
  • Method SET means that resource properties can be updated (or method can be invoked)
  • Method DELETE means that resource can be deleted
  • Method LIVE means that client can subscribe to changes using server sent events (EventSource) - optional feature
  • Method LOCK means that client can lock record so no one else can modify it - optional feature

/contacts

Calling an OPTIONS HTTP method on a /contacts endpoint will return a resource schema which describes the resource model and resource capabilities.

HTTP response:

Content-type: application/json+x.metaapi
{
    "@doctype": "@meta.schema.object",
    "@implements": "@meta.collection",
    "methods": ["GET", "LIVE"],
    "title": "Contacts",
    "properties": {
        "records": {
            "type": "@meta.list.object",
            "label": "Contacts",
            "readonly": true,
            "properties": {
                "id": {
                    "type": "@meta.integer",
                    "label": "Record ID",
                    "private": true,
                    "..comment..": "Private tells UI not to display this field by default."
                },
                "first_name": {
                    "type": "@meta.text",
                    "label": "First name"
                },
                "last_name": "...",
                "email": {
                    "type": "@meta.text.email",
                    "label": "E-mail"
                }
            }
        },
        "count": {
            "type": "@meta.integer",
            "label": "Total count",
            "readonly": true
        },
        "..comment..": "These properties can be accessed within the same resource as this schema using GET, PUT or PATCH methods. In this case of collection only GET method is supported."
    },
    "query": {
        "first_name": {
            "type": "@meta.text",
            "label": "First name"
        },
        "last_name": "...",
        "..comment..": "Specifies query variables that can be used to filter returned properties."
    },
    "relations": {
        "{records.id}": {
            "label": "Record detail"
        },
        "..comment..": "Specifies URI of related endpoints."
    },
    "actions": {
        "create": {
            "label": "Create new contact",
            "..comment..": "Method arguments (properties) can be retrieved by fetching schema for method URI."
        },
        "sendNewsletter": {
            "label": "Send newsletter to contacts"
        }
    }
}

/contacts?offset=10&limit=2

Calling a GET HTTP method on the /contacts endpoint will return a list of contacts.

Because the resource implements a collection interface we know that we can also filter a result and use a pagination.

HTTP response:

Content-type: application/json+x.metaapi
{
    "@doctype": "@meta.data",
    "@implements": "@meta.collection",
    "records": [
        { "id": 1, "first_name": "John", "last_name": "Doe", "email": "john@doe.tld" },
        { "id": 2, "first_name": "Jack", "last_name": "Doe", "email": "jack@doe.tld" }
    ],
    "offset": 10,
    "count": 42
}

/contacts$create

Calling an OPTIONS HTTP method on a /contacts$create endpoint will return a schema of create method of the /contacts resource.

HTTP response:

Content-type: application/json+x.metaapi
{
    "@doctype": "@meta.schema.method",
    "title": "Create contact",
    "methods": ["SET"],
    "properties": {
        "first_name": {
            "type": "@meta.text",
            "label": "First name",
            "required": true
        },
        "last_name": "...",
        "email": {
            "type": "@meta.text.email",
            "label": "E-mail",
            "required": true,
            "hint": "Enter a valid e-mail address."
        },
        "..comment..": "These properties are method arguments. A method endpoint acts as a separated data model."
    }
}

/contacts$create

Calling a PUT HTTP method on the /contacts$create endpoint will create a new contact record.

HTTP request body:

{
    "first_name": "Jiri",
    "last_name": "Hybek",
    "email": "jiri@hybek.cz"
}

HTTP response:

200 OK
Content-type: application/json+x.metaapi
{
    "@doctype": "@meta.response",
    "type": "success",
    "message": "Contact has been successfully created.",
    "data": {
        "id": "42",
        "first_name": "Jiri",
        "last_name": "Hybek",
        "email": "jiri@hybek.cz"
    }
}

/contacts/42

Calling an OPTIONS HTTP method on a /contacts/42 endpoint will return a record schema. It is important to us to be able to retrieve a schema for every resource because we can have a free data structure for records in many cases.

HTTP response:

Content-type: application/json+x.metaapi
{
    "@doctype": "@meta.schema.object",
    "methods": ["GET", "SET", "DELETE", "LIVE", "LOCK"],
    "title": "Contacts",
    "properties": {
        "id": {
            "type": "@meta.integer",
            "label": "Record ID",
            "private": true
        },
        "first_name": {
            "type": "@meta.text",
            "label": "First name"
        },
        "last_name": "...",
        "email": "..."
    }
}

/contacts/42

Calling a GET HTTP method on the /contacts/42 endpoint will return a record data.

HTTP response:

Content-type: application/json+x.metaapi
{
    "@doctype": "@meta.data",
    "id": 42,
    "first_name": "Jiri",
    "last_name": "Hybek",
    "email": "jiri@hybek.cz"
}

/contacts/42

Calling a PUT HTTP method on the /contacts/42 endpoint will update our contact.

HTTP request body:

{
    "first_name": "Jiri",
    "last_name": "Hybek",
    "email": "jiri@hybek.tld"
}

HTTP response:

200 OK
Content-type: application/json+x.metaapi
{
    "@doctype": "@meta.response",
    "type": "success",
    "message": "Contact has been successfully saved.",
    "data": {
        "id": "42",
        "first_name": "Jiri",
        "last_name": "Hybek",
        "email": "jiri@hybek.cz"
    }
}

And so on...

User interface - using META UI

The META UI use XML because it provides great solution for a component model.

/list.meta

The first example is META UI view of the contacts collection.

We define our data source, a list component which is connected to that datasource and some other information.

We can also optionally specify a layout for each record and which fields should be displayed.

Note export attribute which tells client that selected record should be available to another views - see more below.

HTTP response:

Content-type: application/xml+metaui
<?xml version="1.0" encoding="UTF-8"?>
<meta-ui>
    <link rel="nav" src="/nav.meta" />
    <datasource id="collection" src="/contacts" />
    <collection datasource="@collection" filters="true" export="selection:$selection">
        <layout-columns>
            <column>
                <icon src="/media/contact.svg" size="small" />
            </column>
            <column grow>
                <property name="first_name" label="Jméno" />
                <property name="last_name" /><!-- label is automatically gathered from the data source -->
                <link rel="detail" src="/detail.meta?id=#{id}" /> <!-- interpolation passes 'id' property to URL -->
            </column>
        </layout-columns>
    </collection>
</meta-ui>

/contact.meta

This META UI view displays contact detail.

If the resource supports a 'SET' method then properties should be displayed as form fields with a Save button.

Properties can be placed automatically based on the data model or we can specify their placement manually.

HTTP response:

Content-type: application/xml+metaui
<?xml version="1.0" encoding="UTF-8"?>
<meta-ui>
    <link rel="nav" src="/nav.meta" />
    <!-- data source url accepts query argument 'id' -->
    <datasource id="record" src="/contacts/${id}" />
    
    <!-- all properties are rendered automaticaly based on the data source -->
    <properties datasource="@record" />

    <!-- or we can place properties manually -->
    <group title="Personal">
        <property name="@record.first_name" />
        <property name="@record.last_name" />
    </group>
    <group title="Contact" scope="@record">
        <property name="email" />
    </group>
</meta-ui>

/index.meta

Our index page - a META UI view that combines the previous views into a friendly interface with the contact list on a left side and with the contact detail on the other side. We can achieve this behaviour by using an embedding of another META UI views.

Note that we are using exported selection variable from the list.meta view.

HTTP response:

Content-type: application/xml+metaui
<?xml version="1.0" encoding="UTF-8"?>
<meta-ui>
    <link rel="nav" src="/nav.meta" />
    <layout-sidebar>
        <aside>
            <embed src="/list.meta" id="list" />
        </aside>
        <main>
            <embed src="/detail.meta?id=#{@list.selection.id}" />
            <!-- id is gathered from embedded 'list' view which exports it -->
        </main>
    </sidebar>
</meta-ui>

/index.meta

Finally we define a special META UI document called navigation manifest which describes a navigation structure of our application. Client is then capable of displaying that navigation.

Note that this navigation manifest was referenced in the previous views.

HTTP response:

Content-type: application/xml+metaui
<?xml version="1.0" encoding="UTF-8"?>
<meta-nav>
    <title>My app</title>
    <link src="/index.meta" label="Address book" icon="/media/addressbook.svg">
        <!-- child item, client can fetch title automatically -->
        <link src="/list.meta" />
    </link>
</meta-nav>

How to display META UI?

To display META UI views we would need a special web browser or a client application.

In the future if the concept will be popular then I suppose that community can create a production browser. Even better solution would be if current web browsers will be able to implement the META Web.

In a meantime we would like to develop the META Web browser using current web technologies.

META Web use cases

I see the potential of the META Web in many fields. From multiplatform applications, e-commerce solutions, intranet systems, blogs, websites to IoT and embedded devices.

Web and mobile applications

This area doesn't need any comment because it is the primary field and reason why the META Web concept has been created.

Web presentations

Nowadays classic websites look more like web or mobile applications. Even blogs or online shops are form of a web application which communicates with a server and interacts with a user.

But there may be a problem with design and marketing requirements. I'm explaining the issue below.

Web services

As I mentioned in the META API section - web services and applications communicate with each other more and more. Many people say that we are in the API economy era. Also microservices are becoming very popular. THe META Web can provide an effective solution for a service integration.

Internet of Things (IoT) and embedded devices

The IoT is becoming a popular topic and we don't have any standards yet (many companies are working on it).

What if you buy a smart thermostat device which is using the META Web technology? You will get:

  • Desktop / mobile / tablet app - a user interface based on the META UI
  • Possibility to easily integrate the device with another apps or services using the META API

And all this without installing any additional software or studying a vendor specific API documentation.

This also applies for many other devices such as network devices, home data storages, TVs or smart home devices.

Criticism and possible issues

Before you start asking questions read this FAQ first.

Why not to use current web technologies such as Web components and other HTML5 stuff?

One simple reason - performance. Imagine that all code required for the META UI written as a classic web application using HTML, CSS and JavaScript. Then try to use this app on an average smartphone. Ok, it doesn't run much smoothly. And so much code which has to be loaded.

Another argument is use of the META Web in embedded devices which haven't much performance.

And we try to avoid the need to use complex frameworks. Also a native META Web support can bring a better integration to each platform.

Universal concept cannot cover all use cases

That's true. Many various components need to be specified and implemented such as charts, map views, workflow editors or a conversation component for chats.

But the concept counts with very specific needs. The META UI will provide a capability to use standard web technologies to create custom apps which can be integrated into the META UI view. But this won't be mandatory. For example an embedded browser can omit these features due to performance reasons.

It's really bad if we cannot create nice, rich and colorful designs, how can we do marketing stuff then?

Yes! The META Web is killer of graphic designers and marketing. But it's not so hot, really.

When you look at modern websites you can see that they look like mobile applications. They have a minimalistic and clear design. Why? Because many people access web using mobile phones or tablets and a goal of UX designers is to provide comfortable interaction similar to a native mobile platform.

And how a design is made in mobile apps? In a no special way. By creating a meaningful layout and by using nice images and icons. And the META UI provides exactly the same capabilities.

It has no chance to succeed

I partially agree with that. The META Web is the futuristic and maybe utopic concept. But only if its primary goal would be to change a way how web applications are being made and how we interact with the web. But this not means that the META Web would not be usable for effective development of business applications.

Only time will tell.

Conclusion

My goal is to solve problems that bother me as a web developer and I see the META Web as a solution.

This project is being developed as an open-source and I would like to invite other developers to contribute. Maybe right people will find this project and it will become a global web standard.

Personally, I think that if it won't be the META Web then another similar technology will be created in the future which will be solving the same problems.

This manifesto is really only a small introduction to my concept. I'm discuss concept details on a forum and the API and UI specifications will be added over time.

You can also browse an old META Web specification I've been working on. Current work is based on these specs.

How can you contribute?

In the current phase you can help us by discussing the concept on a forum. Two heads are better than one.

You can also help us improve this manifesto and bring new ideas.

Open META Web forum

Subscribe to our newsletter

And be notified about the META Web progress.

* indicates required

Ideas of tomorrow

I would like to list future ideas of the META Web which are not important now.

META Web can also provide:

  • A unified identity between services - Would you like to use the Facebook, Twitter, Google or other services which you have your social profile on to connect your identity to any service? Use the unified META API.

  • Data sharing - The META Web can provide a way how to share data between services or clients - you can fill part of a form then send it to you friend who can fill the rest and can send it back to you.

  • Voice interaction - Applications using the META Web unified API can be easily controlled by a voice, mainly when developer provides enough description in a schema, also the META dictionary can help clients to understand common actions.

  • Artificial intelligence - The META Web service schemas are all machine readable, why not to create an intelligent agent which can interact with your services using intelligence.