Its rather another M2M framework and Push Notification Service considering requirements from low level API access from various IoT platform which works on raw TCP layer.
Quickly understand the basic difference UbiPush with MQTT.
1. UbiPush client does not need any extra protocol stack at client, MQTT needs.
2. In MQTT, SUBSCRIPTION or PUBLISH message is “controlled” from client, in UbiPush, it is at server. It helps to change the message “Route” or “Topic” remotely, dynamically, at server or client, anywhwere.
3.Low power,low size security stack in UbiPush vs SSL in MQTT.
4. MQTT uses multiple channels (socket) for publish and subscribe at client. UbiPush uses single channel.
5. Grouping (multiple subscription) takes large resource in MQTT client, UbiPush does not.
6. Communication between groups (owned by different user) is not designed in MQTT, In UbiPush, it is possible (API not yet released).
7. MQTT is little silent for intermediate complex (CPU, Time) security. Only SSL is the option. UbiPush provides multi-level security based on user need (SSL is not yet implemented) without serious compromising (Hope you understand how to keep secret your Private key sent over mail as all people do, except hardware dongle).
8. MQTT is primarily not a transparent channel. With latest API (V1.7 up), UpiPush can be configured as full duplex transparent medium over a single socket channel.
9. Binary data over MQTT is little tricky (custom MQTT client? And/or File). UbiPush provides straight forward option.
Let us start from middle of technical stuffs, hoping you already know what you are dealing with.
Here, we are talking about Global M2M mesh, where any machine can send message to any machine connected to internet (They call it IoT).
Before you move forward, let me give the example that you need to start “doing” than “reading”
1. Just open putty (Later on you can use any client) and set followings and save the profile (you may need to type again and again) and then Open connection.
Hostname: p2pcluster.org
port: 80
Connection Type: Raw
2. Now, very fast (with in a minute), fire this command in putty console:
Example: TCP:ID:11FE313037C5327CD35BAC6C8D66867F:123
You must get a return statement something like [OK:123]
3. Now open another putty console with same credentials as above and fire command with different ID. Something like
TCP:ID:11FE313037C5327CD35BAC6C8D66867F:1234
You must get the same reply [OK:1234]
4. Great. Now fire bellow command in any putty console (say, at console id-123)
TCP:PUSH:11FE313037C5327CD35BAC6C8D66867F:RID:1234:Hello
5. The “Hello” should be sent to other console (with ID-1234).
Interesting?
6 Now open another putty console with ID 12345.
7. In any Putty console fire below command:
TCP:PUSH:11FE313037C5327CD35BAC6C8D66867F:FRIENDS:Hello ALL
8. You should get “Hello ALL” to all other putty consoles!
Seems interesting? If yes, read further, explore more and put comments or help me to find bugs.
Follow the API to get your own private keys and start using it.
Some preamble:
So, what are the common problems in traditional M2M scenario?
High Level API and custom client stack (MQTT etc?):
I know, you already know Google, Apple, Parse are there giving solution and you are finding ways how to use there API to use in your tiny hardware! Frankly speaking, Its too small tiny hardware and cant afford the high level APIs that those giants are providing for Tablet, mobile, desktops. Why those people cant give simple raw low level API too?? Abstraction? Yes, they have their own business model and security. But we need a low level raw TCP API, and hence, UbiPush is here. It is designed to support ALL KIND of IoT device, starting from a core running raw C or C++ or JAVA or NodeJS or Python…whatever hell running in any OS Android or Windows or Linux or NO-OS. That we intended to do.
ADDRESS:
An IoT device needs at-least a remote IP and a Port to communicate (See OSI model and TCP communication). The IP, the address ! The most precious thing. How will you make your IP static? Or really can you do it? Here, its not a client server communication, rather a client-client communication where all clients are a machine, or individual IoT node or a mobile phone or a PC sitting in deep corner of a LAN or WAN. So, all addresses are virtual and cant assure its static property. How do you handle it? We do some arrangement for case to case basis by fixing stuffs (address as static). I am sure, you always dream a solution like Whatsapp! Anyone can send any message to anyone, anywhere, and you must be waiting when Zuckerberg will release an API extending its XMPP service to allow to connect Whatsapp to your IoT device. Well, I am not his friend, or never will be :P, but I can realize the situation as I am passing with same, so, I made the UbiPush that will relay one machine’s message to another machine independent of its depth of address (local, virtual whatever) and all will be identified by its unique ID, that will be assigned by you (Like your mail-ID, phone no etc. No need to depend on IP4 or IP6 address, I am dealing it at my side. All machines needs to connect with push.ubipush.com at port 8083 and then do send your message to destination indicating remote ID.
ID:
Well, I have removed dependency on dynamic and virtual address and given an ID decided by you. But again, that ID may also not be the best. Did you deploy any IoT device? If yes, you must realize the problems with ID itself. Take an use case, assume you have three devices having ID as 1,2 and 3. And assume you have a logic implemented in ID-1 device, “If I am ON, turn ID-2 OFF”. Very well. Now realize the situation, Device ID-2 became damage and eventually you replaced it by new device (do you give warranty 😛 ?) having ID-4. Oops! Now the old logic “If I am ON, turn ID-2 OFF” will fail ! You need to update the logic as “If I am ON, turn ID-4 OFF”, by updating device ID-1 firmware. Great, I know, till now you are doing so.
Here, is a new concept, that will remove the concept of ID, rater, it is rationalized as various synonymous nomenclature as “FRIEND”, “CHILD” where the device will be classified by a class, not by ID. So, how will be logic? The logic to be implemented as “If I am ON, turn MY-FRIEND OFF” and in Ubipush server, define MY-FRIEND as what-ever device-ID you want to link with ID-2 or ID-4, remotely without actually updating firmware of Device ID-1. Does it make sense? It is there in UbiPush.
And here, a new concept has been added in UbiPush that will help the IoT life much easier. A concept of Grouping has been added. There are two kinds of group- a) Hierarchical Group or Tree and b) Non-Hierarchical or Mesh
What is Group?
Group is a skeleton that stored in server defined and designed by user. User may define and create its own group providing respective Device-ID. When, that particular device is Up (Connected to Push Server), its position will automatically gathered.
a) Hierarchical Tree Group
User can define any node as ROOT (if not defined anything, by default all nodes or devices are ROOT). And can create the child and sub-child and further more. There are no limitation of no of nodes or devices under any node. All nodes at same level are called FRIEND (all Roots are friends, all child nodes under same parent are friends). All sub-nodes under a node are called CHILDS. CHILDS of a node is a different group than CHILDS of an another node. ALLCHILDS is a group that defines all the child and grand-child and grand-grand-childs….so on.
But, what is the hell meaning of this grouping?
Well, a node can send message to any group by simply sending a SINGLE message like:
TCP:PUSH:{KEY}:CHILDS:{DATA}
b)Non-Hierarchical Mesh Group
Non-Hierarchical mesh has its own essence and may desired in many situation than Tree architecture. I personally believe, Mesh can replace and remove the Tree, however, as the TREE concept evolved first and working fine, I am intending to keep it for a while.
Let us assume a real scenario test case: “Turn off all lights in Bed-room” or “Turn off all Garden Lights” or “Turn off all high power Lights” (in Night :P). Interestingly, here we are talking a group of devices and there may be multiple common (not all) devices in various group. So, in practical scenario, a device may be part of multiple GROUP. Hence, the TREE architecture may not suitable for messaging. So, the inclusion of MESH. Here, we can create any numbers of independent group under a single KEY.
How to create MESH group?
I have introducing a new entity called ROUTER that is a completely logical entity which resides in Server. User may configure it and connect any numbers of device with its ID. When any message is thrown targeting that specific ROUTER, the message will be broadcasted all the other device nodes.
Just create ROUTER, and connect device-ID with it using API at section 2.2
Then send your Message using simple TCP API like:
TCP:PUSH:{KEY}:RO:{INTERNAL_ROUTER_ID}:{DATA}
SECURITY:
It is better, I should be silent on this for a while. But,still if you are in hurry, find it at https://wordpress.com/pages/esp8266mesh.wordpress.com/ubipush-m2m-data-security
E-Mail:
How much effort you have given to put IMAP or POP client inside device? Relax! Now the UbiPush can manage all those. Use simple API to send a mail to anybody anywhere.
SMS:
Ubipush supports free SMS using low level simple API. Eventually its available only in India. (Wait for API release)
TIME:
You know how complex to sync your device time, either it is a scheduler or alarm ! Relax, now UbiPush will help you to get the time stamps in possible all formats.
Use API
TCP:TIME::
Example:
TCP:TIME:11FE313037C5327CD35BAC0123456789:+05:30
VIDEO: AUDIO:Transparent Channel: (23/04/2017)
Its a dream of all cloud access people to create and/or get a transparent route to share our high bandwidth data. So, the requirements are pretty well defined. A client-client transparent channel with possibly high bandwidth. Here it is. New API version released with transparent channel where two client need to connect to PushServer and after couple of (precisely two) raw TCP API call, user will get a transparent channel.
The API for RAW data communication is created for high bandwidth Audio/Video transfer. Currently it supports one-one channel. Exemplary client program (C#) under testing. Once tested, shall share in git.
Now, I am tired to type…..I shall update other aspects, now look at the API and try to use it. I need to write more.
And needs testers.
Suggest new API or requirements.
****************** UbiPush API LIST *******************
Version 1.5
Release: 26/03/2017
1. ACCOUNT
All user have to create an account which will be identified by user?s existing e-mail id. The user will be given an unique KEY by which respective operation will be performed.
1.1 CREATE USER ACCOUNT
http://push.ubikeys.com:8083/USER?username=abcd@abc.com&password=1234567890
[username must be a valid email id, password should be of 8 chars in length]
if( username does not exists)
Returns:Mail will be sent for activation and link creation.
Else
Returns: Accound credentials will be e-mailed to user id including KEY
1.2 DELETE USER ACCOUNT
http://push.ubikeys.com:8083/USERDEL?username=abcd@abc.com&password=1234567890
if( username does not exists OR wrong credentials)
Returns:Invalid Credentials
Else
Returns: Accound will be deleted. E-mailed will be sent to user id
1.3 RESET PASSWORD
http://push.ubikeys.com:8083/USERPASSRESET?username=abcd@abc.com&password=1234567890&newpassword=000111234
if( username does not exists OR wrong credentials)
Returns:Invalid Credentials
Else
Returns: E-mailed will be sent to user id with new credentials. KEY will not be changed
1.4 RESET KEY
http://push.ubikeys.com:8083/USERKEYRESET?username=abcd@abc.com&password=1234567890
if( username does not exists OR wrong credentials)
Returns:Invalid Credentials
Else
Returns: E-mailed will be sent to user id with new KEY credentials
2. GROUP
Group is an unique concept in IoT where user can create a hierarchy of devices in a tree structure. It will give a tremendous flexibility to the system and messaging when (if) IFTTT is implemented. And also, in present case, a device may throw message at various level of hierarchy without really having Device ID.
The Grouping can be followed by two ways- a) Hierarchical Tree and? b) Mesh Routing
2.1 Hierarchical Tree:
2.1 .1 GET TREE
http://push.ubikeys.com:8083/GETTREE?KEY=1234567890
http://push.ubikeys.com:8083/GETTREE?KEY=1234567890&NODE=1234
[If NODE ID provided, returns tree from that NODE else returns complete Tree]
Returns: {summary of nodes}Total Tree
2.1.2 ADD NODE or SUB_NODE
http://push.ubikeys.com:8083/ADDNODE?KEY=1234567890&PNODE=1234
http://push.ubikeys.com:8083/ADDNODE?KEY=1234567890&PNODE=1234&NODE=123
[PNODE=Parent Node; NODE=Child Node. Creation of Child node is optional]
Returns: {summary of nodes}Total Tree after addition of Node
2.1.3 DELETE NODETREE
http://push.ubikeys.com:8083/DELNODE?KEY=1234567890&NODE=1234
[All child of NODE-ID 1234 will be removed]
Returns: {summary of nodes}Total Tree after deletion of Node
2.1.4 UPDATE NODE
http://push.ubikeys.com:8083/UPDATENODE?KEY=1234567890&PNODE=1234&NODE=12345
[PNODE-ID will be updated as NODE-ID ]
Returns: {summary of nodes}Total Tree after update of Node
2.1.5 TREE DEPTH
http://push.ubikeys.com:8083/DEPTH?KEY=1234567890
[The depth/level of the nodes in the tree]
Returns: Tree Depth
2.1.6 PARENT
http://push.ubikeys.com:8083/PARENT?KEY=1234567890&NODE=1234
Returns: Parent of Node-ID 1234. Returns ?ROOT? if the child node is the root node.
2.1.7 CHILD
http://push.ubikeys.com:8083/CHILD?KEY=1234567890&NODE=1234
Returns: All child nodes under node-ID 1234
2.2 Mesh Routing
2.2.1 GET INTERNAL ROUTER-NODE LIST
http://push.ubikeys.com:8083/GETINTROUTER?KEY=1234567890
Returns: Internal Router List associated with user ID
2.2.2 ADD NTERNAL ROUTER
http://push.ubikeys.com:8083/ADDINTROUTER?KEY=1234567890&ROID=1234
Returns: Router List after adding Router ID-1234
2.2.3 DELETE NTERNAL ROUTER
http://push.ubikeys.com:8083/DELINTROUTER?KEY=1234567890&ROID=1234
Returns: Remaining Router List after deleting Router ID-1234
2.2.5 ADD NODES AT ROUTER
http://push.ubikeys.com:8083/ADDNODEINTROUTER?KEY=1234567890&ROID=1234&NODE=12345
[Node ID 12345 will be associated with Router ID 1234]
Returns: Final node list after adding link between node-1234 and Router ID-1234
2.2.6 REMOVE NODES FROM ROUTER
http://push.ubikeys.com:8083/DELNODEINTROUTER?KEY=1234567890&ROID=1234&NODE=12345
[Node ID 12345 will be de-associated with Router ID 1234]
Returns: Final node list after removing link between node-1234 and Router ID-1234
3. MESSAGE SENDING
A device or system may send a message to any other device or devices or group by knowing only its ID. The message is supported in raw TCP level and also will be supported in high level including JSON and REST API in near future.
3.1 Register ID
REST API:
http://push.ubikeys.com:8083/REGID?KEY=1234567890&ID=123456
Returns: SUCCESS/FAIL:details of status
Raw TCP API:
TCP:ID:{KEY}:{NODE_ID}
Example#
TCP:ID:4D45EA9CFD7E684A9927EDEBFFE18238:1234
3.2 Get ID
http://push.ubikeys.com:8083/GID?KEY=1234567890
Returns: SUCCESS/FAIL:Details of status
//TCP:GID:4D45EA9CFD7E684A9927EDEBFFE18238
3.3 REMOVE ID
http://push.ubikeys.com:8083/REMID?KEY=1234567890
Returns: SUCCESS/FAIL:details of status
Raw TCP API:
TCP:REMID:{KEY}
Example#
TCP:REMID:4D45EA9CFD7E684A9927EDEBFFE18238
3.4 Push Message
http://push.ubikeys.com:8083/PUSH?KEY=1234567890&RNODE=ALLCHILDS&DATA=%5Bbyte array]
http://push.ubikeys.com:8083/PUSH?KEY=1234567890&RNODE=CHILDS&DATA=%5Bbyte array]
http://push.ubikeys.com:8083/PUSH?KEY=1234567890&RNODE=PARENT&DATA=%5Bbyte array]
http://push.ubikeys.com:8083/PUSH?KEY=1234567890&RNODE=FRIENDS&DATA=%5Bbyte array]
http://push.ubikeys.com:8083/PUSH?KEY=1234567890&RNODE=RID&NODEID=123&DATA=%5Bbyte array]
Returns: SUCCESS/FAIL:details of status
Raw TCP API:
Send Message to all Childs and grand-childs.
TCP:PUSH:{KEY}:ALLCHILDS:{DATA}
Send Message to all Childs.
TCP:PUSH:{KEY}:CHILDS:{DATA}
Send Message to only Parent.
TCP:PUSH:{KEY}:PARENT:{DATA}
Send Message to all nodes only at equal level.
TCP:PUSH:{KEY}:FRIENDS:{DATA}
Send Message to any specific node
TCP:PUSH:{KEY}:RID:{TO_NODE_ID}:{DATA}
Send Message to Internal Router
TCP:PUSH:{KEY}:RO:{INTERNAL_ROUTER_ID}:{DATA}
Example#
TCP:PUSH:4D45EA9CFD7E684A9927EDEBFFE18238:ALLCHILDS:Hello
TCP:PUSH:4D45EA9CFD7E684A9927EDEBFFE18238:CHILDS:Hello
TCP:PUSH:4D45EA9CFD7E684A9927EDEBFFE18238:PARENT:Hello
TCP:PUSH:4D45EA9CFD7E684A9927EDEBFFE18238:FRIENDS:Hello
TCP:PUSH:4D45EA9CFD7E684A9927EDEBFFE18238:RID:1234:Hello
4 MESSAGE RECEIVING
4.1 Message Receive
byte[] will be received as data which is formatted as per user sent by user DATA format
5 MESSAGE MAILING
Send remote recipient e-mail via UbiPush server
TCP:EMAIL:{KEY}:{REM_MAIL_ID}:{SUB}:{DATA_STRING}
6 Get TIME
Get time-stamp in Date, Time Epoch second format with a reference of offset time.
TCP:TIME:
Example:
TCP:TIME:11FE313037C5327CD35BAC6C8D12345F:+05:30
OFFSET must be ?XX:XX format
Returns:
[:()(dd,MM,YYYY)(HH:mm:SS)(EPOCH_SEC)]
Exanple:
[IST:(Wed)(03,22,2017)(23:27:16)(1490205436)]
7 SECURE KEY
Activate Session Key based communication. Client authentication by dynamic Session
Get Challenge (Client/Device Auth)
STCP::ACTIVATE
Returns:
OK:
Key Derivation:
SESSION_KEY= HASH_MD5(KEY+RANDOM_NO)
Use Session key for rest of API as it is.
Get Challenge and Get Response (Server Auth)
STCP::ACTIVATE:
Returns:
OK::
RESPONSE verification: RESPONSE= HASH_MD5(KEY+RANDOM_NO)
De-activate Session Key based communication.
STCP::DEACTIVATE
Returns:
OK:STCP_DEACTIVATED
Use normal key for rest of API as it is.
******************************************************************************
Revision History:
Version 1.5
Release: 26/03/2017
New API:
1. Activate Session Key based communication. Client authentication by dynamic Session
Get Challenge (Client/Device Auth)
STCP::ACTIVATE
Returns:
OK:
Key Derivation:
SESSION_KEY= HASH_MD5(KEY+RANDOM_NO)
Use Session key for rest of API as it is.
2. Get Challenge and Get Response (Server Auth)
STCP::ACTIVATE:
Returns:
OK::
RESPONSE verification: RESPONSE= HASH_MD5(KEY+RANDOM_NO)
3. De-activate Session Key based communication.
STCP::DEACTIVATE
Returns:
OK:
Key Derivation:
SESSION_KEY= MD5(KEY+RANDOM_NO)
Use Session key for rest of API as it is.
Version 1.4
Release: 22/03/2017
New API:
1. Get TIME
TCP:TIME:
Example:
TCP:TIME:11FE313037C5327CD35BAC6C8D12345F:+05:30
OFFSET must be ?XX:XX format
Version 1.3
Release: 20/03/2017
Bug Fixed:
1. Dynamic classification tested (without closing client)
2. Delete Node tested
New API:
1. Mesh Routing added with API:
GET INTERNAL ROUTER-NODE LIST
http://push.ubikeys.com:8083/GETINTROUTER?KEY=1234567890
Returns: Internal Router List associated with user ID
ADD NTERNAL ROUTER
http://push.ubikeys.com:8083/ADDINTROUTER?KEY=1234567890&ROID=1234
Returns: Router List after adding Router ID-1234
DELETE NTERNAL ROUTER
http://push.ubikeys.com:8083/DELINTROUTER?KEY=1234567890&ROID=1234
Returns: Remaining Router List after deleting Router ID-1234
ADD NODES AT ROUTER
http://push.ubikeys.com:8083/ADDNODEINTROUTER?KEY=1234567890&ROID=1234&NODE=12345
[Node ID 12345 will be associated with Router ID 1234]
Returns: Final node list after adding link between node-1234 and Router ID-1234
?
REMOVE NODES FROM ROUTER
http://push.ubikeys.com:8083/DELNODEINTROUTER?KEY=1234567890&ROID=1234&NODE=12345
[Node ID 12345 will be de-associated with Router ID 1234]
Returns: Final node list after removing link between node-1234 and Router ID-1234
2. Message Sending API:
TCP:PUSH:{KEY}:RO:{INTERNAL_ROUTER_ID}:{DATA}
3. API changed:
Old API: TCP:PUSH:{KEY}:{TO_NODE_ID}:{DATA}
New API: TCP:PUSH:{KEY}:RID:{TO_NODE_ID}:{DATA}
4. Email API:
TCP:EMAIL:{KEY}:{REM_MAIL_ID}:{SUB}:{DATA_STRING}
V1.2-17/03/2017
Bug fixed:
1. TCP:PUSH:{KEY}:CHILDS:{DATA} API has been verified
V1.1-17/03/2017
Bug fixed:
1. Multiple ID assignment to single client
Configuration:
1. ID assignment time out changed to 60 sec
2. SYNC time changed to 5 mins
esp8266 mesh WIFI 