Palmetto

Palmetto makes it easy for apps to share data across the web, all while giving control and privacy to the user. Palmetto does not actually store any data; it's simply a way to define how data can be accessed across systems.

By curating common data across systems, we enable users and vendors to create deep connections, enabling a strong understanding of trust, which can be used for permission granting and discovering new information.

This secure and distributed system is comprised of two components:

The Palmetto API is a crowd-sourced gateway for APIs across the internet, allowing any app to communicate with other apps based on common data attributes. In case you aren't familiar, an API is a way for two websites to share data automatically, regardless of which programming language each uses, and regardless of how the data is labeled (see background below for a greater explanation).

The Palmetto ID protocol describes how online identity providers help users control their data by interacting with the Palmetto API. This gives users the ability to seamlessly administer all of their data - and set permissions - in one central location.

Palmetto is led by a collection of web enthusiasts who believe opening up data online, and giving complete control to the user, will enable entrepreneurs, educators, and thought-leaders to do some amazing good for society. Similar to Wikipedia, our mission is to use the power of the crowd to curate data. This effort is completely not-for-profit and we encourage everyone to participate.

Goals of Palmetto

→   Allow data to move between apps regardless of programming language;
→   Highlight the best people and apps on the web through collective reputation;
→   Create a safer web by making all interactions secure;
→   Give users control over their data via Identity Providers;
→   Make finding information easier by analyzing signals across the web;
→   Spur innovation by giving entrepreneurs access to users and commodity services;

Instead of thinking of the web as a collection of websites, we need to think in terms of common data. This includes user accounts (profile info), status updates, message threads, pictures and videos, comments and ratings, physical check-ins, and lots more. These are the core features which comprise the majority of our interactions online.

Websites simply provide the means to interact with - and present - our data. The way our data is organized online right now leads to a myriad of problems like duplication of content, lack of control, lack of privacy, lack of security, scaling issues for vendors, unnecessary barriers to entry (which stifles innovation), and much more.

By re-aligning the balance of power over our data (from vendors to users) we can realize the full potential offered by the connectedness of the Web.

Through the objectives of Palmetto, we intend to: advance the cause of open data; provide much greater control, privacy, and security to end users; build a wealth of reputation history; allow entrepreneurs and developers to innovate quicker and easier; reduce worries about scaling and availability of services; allow vendors to focus on product features and customer service over lock-in strategies.

There Are Two Components of Palmetto

→ The Palmetto ID protocol defines how user identities work with Apps and Data;
→ The Palmetto API is a crowd-sourced index of API Data available on the Web;

The problem is, there can't be a protocol for Data; only APIs.

And with good reason. Developers are proud, autonomous beings; we take pride in our language and thrive on solving complex problems in novel ways. Being told there's only one way to write a program is not going to happen. We all currently agree to network and markup standards (namely HTTP and HTML/CSS) because the incentive is there: it's the least amount of work to reach our audience.

Understanding the state of data - and how developers operate - we realize we need a layer on top of the massive amounts of Data around the web. A layer that organizes and translates Data based on their common attributes. This layer is called the Palmetto API.

We can, however, use a protocol for handling online identities better.

There have been many brilliant attempts at solving this problem over the past half decade. Some extremely bright visionaries have created in-depth specifications that would work. The problem, again, comes down to incentives. This time it's the users. End users - the average, daily users who want to watch videos, read news, and communicate with friends - have no incentive to go out of their way to use a more sophisticated identity system. Even with all the buzz in the news about privacy issues and the frustration of duplicate services…. end users just want their content.

We, as developers and system architects, need to realize the end-users' lack of incentive and create a system that "just works," while simultaneously solving the current issues caused by the lack of unified identity. This is the purpose of the Palmetto ID protocol.

Some Background To Understand How This Works

If you aren't familiar, an API is like a language translator for computer systems.

APIs allow two applications to talk to each other (share data), regardless of what programming language is used for each app. This is accomplished by defining common vocabularies and a common data transport method. Analogous to spoken language: if one app speaks German and another speaks Spanish, an API will describe how the two can easily communicate. Except instead of German and Spanish it's Ruby and Python or PHP and Scala or Java and C++ or any other combination of programming languages.

Application developers are free to label their internal data attributes however they'd like. They simply need to map the translation to the API's requirement. For example: a photo sharing app's API might call a "picture" a "photo." When a developer plugs into that API, they need to specify which field on their system corresponds to the "photo" field required by the external app's API.

When you boil down data, everything has common attributes.

Sticking with the picture analogy, a picture contains common attributes across the board like title, description, date and time taken, resolution, physical geo location where the picture was taken, people located in the picture (along with box coordinates), the camera used, etc.. An API is meant to map the vocabularies between two systems based on these common data attributes, giving developers working on different systems the ability to add, edit, read, and delete the data.

APIs are wonderful things. They have helped spread data around the internet, and provided access to commodity based technologies that developers can use to "plug-and-play" rather than having to recreate that functionality - see Twilio as a good example (offering plug-and-play telephony for developers to use in their apps).

The proliferation of APIs across the web, while great, has created an unforeseen complication. Each API's transport method - and vocabularies for common data - are defined by the owners of each independent system. This has caused segmentation of vocabularies. Developers wanting to use these APIs must either write translations for each API, or pick one provider. Due to development constraints, most projects opt for the latter method of picking one provider and only using their API. This has inadvertently lead to the creation of huge data silos.

Solving this problem was the catalyst that brought Palmetto into thought.

Since specification began, we realized that a distributed, open data construct could simultaneously solve a few more issues being experienced on the Web.

In conjunction through the Palmetto ID protocol and the Palmetto API, there are five primary issues addressed by Palmetto:

∠ Identity (Establishing Trust & Software) ∠ Open Data ∠ Context ∠ Security ∠ Privacy

Issues Being Addressed & Benefits

Issue 1→ Identity, part 1: Establishing Trust

You must recognize that identity is deeper than your name and digital profile. Identity is comprised of your relationships and possessions - online that equates to your interaction history with other users and apps, along with the data you create along the way.

It's important to understand why online identities matter: reputation - which can be used for granting permission to your data, and deciding who's opinion to trust.

So the goal of reputation online is to facilitate trust. And why does trust matter? Trust guides us through our daily lives. Trust tells us whom to allow in our homes, where to buy our food, which lands to visit, and much more. Trust is built through interactions with others - be it other people, companies, or products.

The closer we are to the experience, the more weight that trust carries. Collective trust, expressed as reputation, has proven to be extremely valuable online. Look at seller ratings and product reviews on eBay and Amazon, respectively. Also look at the extremely useful voting systems for Q&As on StackOverflow and Quora. For more, check out Reputation Systems on Wikipedia.

We don't think about trust often because it's hard-wired into our brains. Humans have evolved this way. It can't be changed overnight. This is just the way we work. It's very important to recognize this as we continue to shift our personal lives online. We need to build systems that match. That's one of the main goals of Palmetto.

The system we have online right now works against our instincts.

Back when the World Wide Web started, it was simply a network of documents with information. The publisher of each document could put anything they wanted on their page. We had no quick and easy way of determining the veracity of their claims. Then, Google (and others) started analyzing hyperlinks between pages as a mechanism to gauge trust. The more links a site has from other reputable sites, the higher the level of trust, and the higher the placement in the search results.

This worked great for awhile, but we've hit a wall. Our identities -and data- has become locked up in silos. Some of these companies sell our private information to marketing firms. It's been a necessary trade-off that has run it's course. The Web is no longer a passive network of basic knowledge. It has grown into an integral part of the functioning of our daily lives. In order to protect ourselves, and foster future innovation, we must shift the priority to users.

Issue 1→ Identity, part 2: Software

For identity to work, we need a unifying protocol which helps establish trust between a network of people, apps, and data, relative to each node currently interacting with the system, regardless of their point of access (the piece of software or website they are using).

We need to escape from the mentality that an indefinite URL is our identity. This was required for awhile, but now that we're so interconnected - and becoming exponentially more so - we can start to think of our identity as a composition of our data and relationships. Much like when we change our phone number, we still maintain our relationships and possessions. We simply need to inform all relevant parties of our new contact information. Our online identities should work the same way.

We will have an ID that represents our online self. Just like we have a phone number. We simply need to point the apps we use, and the people we communicate with, to the new number upon a switch. The ID we use will be provided to us by our ID provider, much like a phone number is provided to us today. That ID provider will be responsible for making sure other ID providers can find us, based on standard protocols… again, analogous to traditional telcos.

Your canonical online ID does not need to be static forever. Just like a phone number, if you switch ID providers, you must tell all of your contacts and companies you do business with the new way to reach you. This will be a standard feature of all ID providers.

What we need instead is a super simple communication protocol.

A simple protocol used by Identity Providers to facilitate communication between users and apps, and allow for the remaining components to operate.

ID Providers are not forced to use any specific programming syntax or database structure (as long as they follow the protocol). That being said, a major goal will be to create open-source implementations of a typical ID Provider in multiple languages, hosted on Github. This way, anyone can create their own ID provider. There are definite advantages to using a dedicated ID Provider, like availability and redundancy, but we must keep the system completely distributed.

Issue 2→ Open Data (Data Interoperability)

Commoditized web services makes it easier for entrepreneurs to innovate.

In Doc Searls's excellent piece, Independent Identity, he writes:

In a truly free marketplace, vendors in a category compete openly for a customer's business, based on information the customer supplies at his or her discretion to any or all of them. Customer data isn't isolated in vendor silos, and customers aren't forced to go from one silo to another and interact separately with each vendor's customer relationship management system, its lame marketing agendas and its locked-up data.

Think about how a Number 2 Pencil is made: it’s made of wood and graphite and rubber and metal. These are the features common to all pencils. Thanks to the division of labor, it’s very easy for many different types of pencils to be manufactured based on these core components. Separate companies specialize in one area of the construction of a pencil. Each cog in the wheel works harmoniously with the others to build a great final product, with many more options than if one company cornered the market.

Try to imagine if there were only one graphite mining company who would not sell graphite to other companies to build pencils. Instead, imagine if they decided they were the only company who would build and sell pencils. This would be great for them – corner the market and make tons of profit – but terrible for the consumer.

We believe the best way out of the silo'd web is to create a crowd-sourced API index of all other APIs available on the internet. This is the heart of Palmetto; define a method for data interaction which puts users in control of their data, and incentivizes application owners to provide access, enabling a true free market of commoditized data services.

Businesses should focus on their product and customer service; not blocking competition.

Businesses should be rewarded on the basis of merit (quality of their product and customer service), not on their ability to lock-in customers. By creating and using decentralized commodity services on the web, entrepreneurs can focus on building better, more innovative products that will help make everyone’s lives easier, rather than focusing on how to block their competition and force their users into accepting their less-than-sufficient product.

In the decentralized web, engineers are able to build incredible information discovery services. These systems, combined with the ability to link up to contacts across systems, will make bringing a new product to market easier than ever. Good apps built by great people will naturally rise to the top. The person with the most money to market their product will no longer automatically win.

Scaling and availability is less of an issue.

Separating the core features which comprise most web apps leads to multiple companies serving the same market, providing a fail safe in case one of them experiences connectivity issues. For example: if your current provider of pictures goes down due to a freak weather occurrence in one part of the world, your pictures will automatically be served from an alternative provider. The applications you subscribe to which use your pictures are able to work with any provider automatically and instantaneously.

Information is allowed to truly proliferate.

A system of distributed, mirrored services can’t be shut down by governments as easily. There is still an issue with centrally controlled DNS systems, but we have removed at least one more bottleneck. By extending the network effect to all apps across the web, innovation within education can spread like a wildfire. Educators and social-do-gooders are able to reach the under privileged much easier, providing access to information and knowledge like never before.

Making society operate safer and more efficiently.

Anonymous usage data helps make government more efficient and provide deep insights into developing markets, leading to incredible innovation.

There is a wealth of information on how society works that is locked up in various systems across the internet. This information, if unlocked and allowed to be analyzed, can lead to some amazingly innovative and efficient new products and services that will make life much easier.

Issue 3→ Context (Semantic Data)

And how do we determine reputation to facilitate trust? Context.

In order to value relationships between users, apps, and data, there are two things we need to establish. First, we need to identify the characteristics which comprise context, and second we need to figure out if there is a way to systematically / programmatically determine context based on those characteristics.

The characteristics of context are analogous to basic real-world interactions: who, what, when, and where. The missing “why” is not important technically; that’s up to the human practitioners and viewers to decide. The missing “how” is the context we are trying to determine.

The characteristics of context as it relates to the users, apps, and data.

→ Who – the users and apps currently involved in any interaction;
→ What – the data involved in the interaction;
→ When – the chronological relationship of the interaction;
→ Where – geographical relationship between the participants (both online and offline);

These pieces are fairly easy to determine, technically. The Palmetto API provides access to the signals, but the real driver to determine semantics is the “how.” How do these characteristics relate to one another?

The key is in the frequency of interactions combined with reputation of the participants.

And it’s all relative. How many times a user interacts with another user – within a set of users – will determine their relationship. Same goes for how often users interact with apps and data. This also counts for how often apps communicate with other apps and data. The implicit reputation based on the frequency of interactions is fairly easy to quantify. The difficult part is determining the explicit reputation each user or app gives to other users, apps, and data. This has been the missing puzzle piece until now; the Palmetto API is the mechanism that allows users to easily rate all other participants’ actions – along with data and apps – within the system.

There are very successful case studies of reputation at work online (see Amazon and eBay).

By defining common data, and providing a way for apps to interact with one another regardless of language, we can expand the benefits of user reputation. The explicit reputation given by users, in conjunction with the automatic implicit reputation based on the signals of who, what, when, and where, allows us to effectively, systematically determine context and discover real semantic meaning.

Issue 4→ Privacy

In the decentralized web, the user is the center of attention.

All apps must communicate with the user to learn which services they use for various common data (pictures, videos, status updates, etc). This gives users the ability to decide exactly who has access to which parts of their data. In some cases sharing your data with a web app is great. It can lead to a very personalized experience, making your life much better (through product discovery for example). In other cases, you don’t want companies to have access to your personal data (like spammers for example). By shifting control from the service providers to the individual users, we are able to hold the service providers accountable, and make the web a much better experience.

Making data interoperable means Reputation can be used to protect our data and recommend services.

When an app has information about how a user's data relates to all of their contacts and vendors, we can start assessing reputation to make decisions about which systems to trust and other services to recommend. Instead of relying on a few - possibly corrupt - entities to validate the trust of a service, we can instead ask our trusted contacts and vendors who they trust. This is how the real world works and it's how the online world should work.

By analyzing our usage patterns, along with the usage patterns of our contacts, application architects can develop algorithms that find the most relevant data at all times; in most cases we will no longer have to search for information we want; it will automatically be presented to us.

Issue 5→ Security

Security and control over user information will be at the forefront; not an afterthought.

We are security zealots and have taken every attack method into consideration. All transactions processed via the Palmetto API require the highest known secure transfer protocol. Likewise, the Palmetto ID protocol requires robust symmetrical authentication during every request. The last piece of the security equation uses the relative trust of your peers to validate the participants in every transaction are who they say they are.

Some Pragmatism Regarding User Adoption

If you're still reading, you're probably a developer or entrepreneur saying, "yeah, great, we've heard this all before, but you'll never gain user adoption." And you're right to think that. The specifications for this system are fairly easy. The implementation is Mt. Everest.

Gaining user adoption: Ease of Use

*Explanation soon*

Gaining user adoption: Simple Protocol; Powerful Apps

The protocol is dead simple. It's up to the developers and system architects to build deep logic into their apps.

Gaining user adoption: Distributed

*Explanation soon*

Gaining user adoption: Profile ID Propagation

When you change your ID, it propagates to all of your contacts and vendors automatically (to their ID providers).

Gaining user adoption: Requiring Participation

The Palmetto API works in a share-alike environment. In order to retrieve data from other apps, you must be willing to share that same data. The Palmetto API routinely verifies participation of all apps and maintains a "selfish" list other apps can block if necessary. Additionally, all apps must support authentication via the Palmetto ID Protocol.

A Few Practical Use Cases

Here are a few common use cases that demonstrate the flow of a typical transaction. Please suggest more possible scenarios to help think critically through all aspects of the API specification. See Get Involved below for contact info. Thanks.