Binary-Safe SMS HTTP API v0.9.7

1. Introduction

HORISEN Binary-safe SMS HTTP API is a variation of SMS HTTP API (please check Reference 1.).

Standard SMS HTTP API is designed for A2P traffic, for integration with applications that need as simple as possible high-level API just to send SMS, and because of this, the API has some limitations:

  • It is possible to send only GSM and UTF-16 messages, not binary,
  • It is possible to send concatenated SMS as one request (only) and the API takes care of splitting a long SMS into SMS concatenated parts (Note: this is not available in Binary-safe API),
  • There is a certain number of fields in the SMS that are filled automatically by the API.

This document describes a proposed extension, Binary-safe SMS HTTP API, that overcomes these limitations by exposing all SMPP fields of submit_sm and deliver_sm PDUs over the HTTP JSON API. This also means that the customer’s familiarity with SMPP PDU, as defined in the specifications (link is in References):

  • SMPP v3.4
  • SMPP v5.0

is needed to properly format SMS MT submissions and to decode received DLR and SMS MO notifications.

This document does not explain the usage of SMPP fields.

Binary-safe API leaves formatting of the "short_message" field (smppShortMessage), UDH, and the other fields to the customer. This means that a customer has full control over all fields in SMPP PDUs, including fields such as "data_coding" (smppDataCoding) and "esm_class" (smppEsmClass).

2. Terminology

This chapter provides information about the terminology that is in use:

  • API is HTTP SMS API extended with support for Binary-safe SMS submissions.
  • SMS Platform - Platform for transit of SMS messages that the API is a part of.
  • Customer is the party that implements a business application that uses Binary-safe SMS API and provides callbacks for receiving DLR and MO notifications.
  • Supplier is the company that provides SMPP/SS7 connections used by the SMS Platform to submit MT messages to the destination device and to receive DLR and MO notifications.
  • Supplier gate is an artifact in SMS Platform that denotes the 'destination of the SMS' - it is usually connected to one Supplier SMSC using one account, although alternative setups are possible. For details, please check SMS Platform documentation.
  • The callback is the URL (HTTP or HTTPS) provided by the Customer where API submits DLR and MO notifications.

3. Protocol

The customer communicates with the API using HTTPS protocol, POST method with content type 'application/json', and JSON-encoded payload. UTF-8 encoding must be used when encoding and decoding JSON messages.

API provides endpoints for:

  • Sending MT messages from the customer.

API uses callbacks for:

  • Sending DLR notifications to the customer,
  • Sending MO notifications to the customer.

The customer should provide a different URL for both callbacks to have this configured on the API.

4. Authentication

4.1 Authenticating Submissions from the Customer

Submissions (MT messages) from the customer are authenticated by the API via a username and a password.

4.2 Authentication Callbacks to the Customer

Current implementation of the callbacks can be authenticated on the customer side using HTTP basic authentication.

Parameters for authentication (username/password) can be set in the account settings.

5. General Naming Convention

API follows the naming convention:

  • All the fields that correspond to matching SMPP fields are in camelCase with the prefix 'smpp',
  • All the fields that are API-related but have no correspondence in SMPP fields are in camelCase.

Some fields can be sent in one of the supported formats, and the name suffix denotes the format.

Data type Suffix Format Example
int (none) decimal integer "smppDataCoding": 8 1)
int Hex hex integer "smppDataCodingHex": "08" 2)
string (none) string "smppShortMessage": "This is a string" 3)
string Hex string in hex format, 2 digits for each string byte "smppShortMessageHex": "54686973206973206120737472696e67" 3)
string Base64 string encoded in base64 encoding "smppShortMessageBase64": "VGhpcyBpcyBhIHN0cmluZw==" 3)

Notes for the previous examples:

1) Note that here we have integer JSON data type, without double quotation marks.
2) Note that here we have integer represented with JSON string data type (enclosed in double quotation marks), and it contains hex representation of the integer data type.
3) Note that here we use string representation, which by JSON encoding standards must be enclosed in double quotation marks.

The rules defined above are due to the strict way how GoLang parses JSON. Some other programming languages allow JSON integer types to be sent as strings as well (e.g. "smppDataCoding": "8") but this is inconsistent with JSON definition.

In the examples above:

  • "smppDataCoding" and "smppDataCodingHex" are alternative ways to send the same value but using a different format,
  • "smppShortMessage", "smppShortMessageHex" and "smppShortMessageBase64" are alternative ways to send the same value but using a different format.

How to encode various types:

  • Integer is best to be encoded as JSON type integer, without double quote marks. Exception to this rule should be when encoding it as hex gives more clarity.
  • String is best to be encoded as JSON type string, in any of the string (plain), hex or base64 format.
  • Strings that are more clearly represented as array of bytes (e.g. "smppShortMessage"), because content may be GSM-encoded string, UTF-16 string or binary message are best to be encoded either in hex or base64 format.

For example:

  • "smppEsmClass": 1
  • "smppValidityPeriod": "220119071435004+"
  • "smppShortMessageHex": "5468697320697320612074657374206d657373616765"
  • "smppShortMessageBase64": "VGhpcyBpcyBhIHRleHQgbWVzc2FnZQ=="

One exception is the field "smppSmLength" which is always calculated to match "smppShortMessage" length in bytes.

5.1 Fields with Int Data Type

Value for each field of data type 'int' can be passed either as JSON integer type or as a string where value is encoded in hex. Value is encoded with two hex digits. It is important when formatting JSON to submit integer fields (when given in 'int' format) without quotation marks.

E.g. let’s encode "smppEsmClass" value 64 (hex 0x40):

Correct:

  • "smppEsmClass": 64
  • "smppEsmClassHex": "40"

Incorrect:

  • "smppEsmClass": "64"

5.2 Fields with String Data Type

Values of data type 'string' can be passed either as a plain string (utf8 encoded), hex representation of byte array, or base64 representation of byte array. For example:

  • "smppShortMessage": "This is a text message"
  • "smppShortMessageHex": "5468697320697320612074657874206d657373616765"
  • "smppShortMessageBase64": "VGhpcyBpcyBhIHRleHQgbWVzc2FnZQ=="

are equivalents.

API will not change or convert any of the submitted fields (except where noted, like in the case of "smppSmLength" field), so there is a guarantee that the customer can have full control over the encodings and binary messages.

The disadvantage of sending hex or base64 representation is that, especially in the case of "smppShortMessage" field where the customer must encode the content of this field to match "smppDataCoding" value, it is not trivial to encode the content.

6. Submitting MT Messages

URL: https://sms.horisen.info:12021/bulk/sendsms-binsafe
Method: POST
Content-type: application/json
Payload: a JSON object with the structure described below. It should be UTF-8 encoded.

6.1 JSON Payload

The JSON payload has the following structure (full example):

{
    "auth": {
        "username": "13320_HTTP_TEST",
        "password": "somepassword"
     },
    "dlrUrl": "https://customer-host/dlr-notification.php",
    "smppServiceType": "",
    "smppSourceAddrTon": 5,
    "smppSourceAddrNpi": 0,
    "smppSourceAddr": "test",
    "smppDestAddrTon": 1,
    "smppDestAddrNpi": 1,
    "smppDestinationAddr": "41787078880",
    "smppEsmClass": 3,
    "smppProtocolId": 0,
    "smppPriorityFlag": 0,
    "smppScheduleDeliveryTime": "",
    "smppValidityPeriod": "",
    "smppRegisteredDelivery": 1,
    "smppReplaceIfPresentFlag": 0,
    "smppDataCoding": 0,
    "smppSmDefaultMsgId": 0,
    "smppSmLength": 0,
    "smppShortMessageBase64": "AFjnGjaASzuucs0SvQAAQFZK+YzT1Ru7Lqppw/0lDBlH1G3O9JcxQSgwoQAABDrHONG0xCc1wASAAAAAAAAAAAAAAAAAAAAAAAAAAByrsmdXn3PTRVnQed/BR+J1ykkmVShZJl1NA6iUDfxX+LasTXo9I3vsXbJBtuy5diaRPm87d6LxgEz0Z+EpBgA=",
    "tlv": [
        {
            "tag": "020b",
            "value": "1234"
        },
        {
            "tag": "40cf",
            "value": "ab8973cf"
        }
    ]
}

API allows some fields to be omitted from the JSON, and they are then given default values by the API.

6.2 SMPP Fields Presented in the SMS MT Object

The following table documents SMPP fields present in the SMS MT object.

Name Data type Supported format Mandatory Default value
smppServiceType string string, hex, base64 no (empty)
smppSourceAddrTon int int, hex no auto-discovered by the platform
smppSourceAddrNpi int int, hex no auto-discovered by the platform
smppSourceAddr string string, hex, base64 yes /
smppDestAddrTon int int, hex no auto-discovered by the platform
smppDestAddrNpi int int, hex no auto-discovered by the platform
smppDestinationAddress string string, hex, base64 yes /
smppEsmClass int int, hex no assembled by the platform, 0x03
smppProtocolId int int, hex no 0
smppPriorityFlag int int, hex no 0
smppScheduleDeliveryTime string string, hex, base64 no It can be empty.
smppValidityPeriod string string, hex, base64 no 24h in absolute format
smppRegisteredDelivery int int, hex no 1
smppDataCoding int int, hex no 0
smppSmDefaultMsgId int int, hex no 0
smppSmLength int int, hex no calculated based on smppShortMessage
smppShortMessage string string, hex, base64 yes /

TLVs are given as pairs of values ("tag", "value") and for both hex content is assumed, they must be sent without format suffix Hex.

Additionally, JSON contains fields:

  • "auth" - a sub-object with the properties:

    • "username"
    • "password"
  • "dlrUrl" - a callback URL (HTTP or HTTPS) for sending DLR notifications described in the section 7. Sending DLR Reports.

The rest of the fields are SMPP fields, as described in the document SMPP Protocol Specification 3.4 issue 1.2 from the references.

6.3 Shorter Example, with most of the optional fields omitted

{
   "auth": {
      "username": "13320_HTTP_TEST",
      "password": "somepassword"
    },
   "dlrUrl": "https://customer-host/dlr-notification.php",
   "smppSourceAddr": "72589",
   "smppDestinationAddr": "+4915178950119",
   "smppDataCoding": 247,
   "smppRegisteredDelivery": 1,
   "smppShortMessageBase64": "AFjnGjaASzuucs0SvQAAQFZK+YzT1Ru7Lqppw/0lDBlH1G3O9JcxQSgwoQAABDrHONG0xCc1wASAAAAAAAAAAAAAAAAAAAAAAAAAAByrsmdXn3PTRVnQed/BR+J1ykkmVShZJl1NA6iUDfxX+LasTXo9I3vsXbJBtuy5diaRPm87d6LxgEz0Z+EpBgA="
}

6.4 GSM 03.38 Encoded Messages

If the "smppDataCoding" value is 0 (with the convention that on most systems 0 means GSM 03.38 encoding) the field "smppShortMessage" should contain a hex or base64 representation of GSM 03.38 encoded string.

6.5 Unicode Messages

If "smppDataCoding" value is 8, customers must encode "smppShortMessage" as UCS2-BigEdian or UTF-16, depending on what encoding the supplier gate understands as encoding 8.

Standard explains that "smppDataCoding" value 8 says that message is Unicode, but implementations on the supplier side and destination devices may vary, depending on what they expect.

The standard says that "smppShortMessage" is then UCS2/Unicode-16 (each character as 2-byte Unicode-16 encoding, in big-endian format), but more and more systems and devices understand this value as UTF-16, mainly to be able to support the rest of the Unicode ranges that are not present in UCS2/Unicode-16, such as emoji characters.

6.6 Binary Messages

The field "smppDataCoding" with value 4, by standard, denotes binary SMS. In the case of binary messages, hex or base64 representation should be always used.

6.7 UDH

UDH is encoded in the "smppShortMessage" field as described on the following page wikipedia.org/wiki/User_Data_Header and requires "smppEsmClass" with UDHI bit set. It is also recommended, when SMS has UDH, to submit also "smppShortMessage" field either as "smppShortMessageHex" or "smppShortMessageBase64".

6.8 Long SMS

A long SMS, that is delivered to the destination device as series of concatenated message parts, must have concatenation information encoded in the UDH, as described on the page wikipedia.org/wiki/Concatenated_SMS and must be sent in separate submissions, one for each Concatenated SMS part. Concatenation information is encoded in UDH field, with UDHI bit in "smppEsmClass" bit set (please refer to SMPP v3.4 documentation).

6.9 Optional Fields (TLV - TAG-LENGTH-VALUE)

The SMPP protocol allows optional fields to be attached to each PDU, as described in SMPP Protocol Specification 3.4 issue 1.2 document. These fields can be added to an SMS MT submission by adding an array of objects in the field "tlv". Each object has "tag" and "value" encoded always as a hex representation of a byte array. The length of the TLV is always calculated, so it doesn’t have to be submitted by the customer.

6.10 HTTP Response on MT Submission

The HTTP response on MT submission can have two possible outcomes - accepted and rejected submission.

6.10.1 HTTP Response on Accepted Submission

In the case of accepted submission, the response is with status code 202 (or anything from the range 2xx), content type is 'application/json', and JSON content is:

{
    "msgId": "9325d0a8-2638-11e6-afe7-bffc7cc8fa4f"
}

This message ID should be saved by the customer, because DLR notifications will send this field to denote for which SMS submission DLR notification is sent to the client.

6.10.2 HTTP Response on Rejected Submission

In this case HTTP response can be:

  1. With the status 4xx
  2. With the status 5xx

In the case of the status code 4xx, the body contains JSON document with the error, such as:

{
    "error": {
        "code": "107",
        "message": "Invalid sender"
    }
}

Error codes are listed in the document HTTP Bulk Service API in 2.4 Error codes section.

In the case of the status code 5xx, the body should be ignored. There is no guarantee that it can be parsed as JSON or that it is machine-readable.

7. Sending DLR Reports

Delivery reports for each SMS MT are sent to the callback defined in "dlrUrl" field of the SMS, as POST method with 'application/json' content type and JSON-encoded content using UTF-8 encoding.

Except "smppShortMessage" always sent in both string and base64 format, other fields are sent in the base data type (integer or string).

DLR is always sent to the customer with all the fields included. Typically, SMPP fields are informal, all the needed data for tracking the delivery of the MT message are in JSON fields.

7.1 Correspondence with SMPP Protocol

In SMPP protocol DLR and MO messages are typically received from the supplier as deliver_sm SMPP PDU. This PDU is multi-purpose, and besides DLRs and MO messages in some scenarios are used for sending MT or P2P messages. Therefore, this PDU have fields that have no semantic meaning in DLR nor MO, but sometimes are used by suppliers. They are present in JSON payload of these callbacks for completeness, Customer will in most cases ignore them.

7.2 TLV Fields in DLRs

The format of DLRs is not strictly standardized in SMPP. There are recommendations on how to implement them, but they have minor differences.

For example, delivery status may be determined from smppShortMessage field of DLR or from TLV field message_state. The precedence of one or another is not clearly defined.

SMS Platform applies the same algorithm as it is using for SMPP, on Binary-safe API to extract the following information from the DLR:

  • Delivery status
  • Delivery status code (numeric representation of delivery status)
  • Error Code (after error code unification to the common error code values)
  • SMSC error code (original error code as received from the supplier)
  • SMSC error code NACK (original status taken from the submit_sm_resp PDU).

These fields are provided as a simpler way to discover the outcome of delivery, but nevertheless, API always attaches all the fields, including TLVs, as they are received from the supplier.

TLV subsection is optional. If no TLVs are present it will be omitted.

7.3 Example

Example of DLR notification payload:

{
   "msgId": "9ceec996-4014-9bfd-8019-0891f8e799c8",
   "accountName": "200114_HTTP_Test",
   "deliveryStatus": "delivered",
   "deliveryStatusCode": 2,
   "errorCode": 0,
   "smscErrorCode": 0,
   "smscErrorCodeNack": 0,
   "smppServiceType": "",
   "smppSourceAddrTon": 1,
   "smppSourceAddrNpi": 1,
   "smppSourceAddr": "4915178950119",
   "smppDestAddrTon": 0,
   "smppDestAddrNpi": 1,
   "smppDestinationAddr": "72589",
   "smppEsmClass": 4,
   "smppProtocolId": 0,
   "smppPriorityFlag": 0,
   "smppScheduleDeliveryTime": "",
   "smppValidityPeriod": "",
   "smppRegisteredDelivery": 0,
   "smppReplaceIfPresentFlag": 0,
   "smppDataCoding": 0,
   "smppSmDefaultMsgId": 0,
   "smppSmLength": 128,
   "smppShortMessage": "id:9ceec996-4014-9bfd-8019-0891f8e799c8 sub:001 dlvrd:001 submit date:2212161247 done date:2212161247 stat:DELIVRD err:000 text:",
   "smppShortMessageBase64": "aWQ6OWNlZWM5OTYtNDAxNC05YmZkLTgwMTktMDg5MWY4ZTc5OWM4IHN1YjowMDEgZGx2cmQ6MDAxIHN1Ym1pdCBkYXRlOjIyMTIxNjEyNDcgZG9uZSBkYXRlOjIyMTIxNjEyNDcgc3RhdDpERUxJVlJEIGVycjowMDAgdGV4dDo",
    "tlv": [...]
}

DLR, besides SMPP PDU fields, additionally contains:

  • "msgId" field - the same one that is returned in the HTTP response when the MT was submitted.
  • "deliveryStatusCode" field, with values from SMPP Protocol Specification 3.4 issue 1.2 document, section '5.3.2.35 message_state'. Presented in the table below in 7.8 section.
  • "errorCode" - if message is rejected by the platform or the supplier gate. Error code is translated using error code translation table assigned to the gate.
  • "smscErrorCodeNack" - if message is rejected by the supplier gate using "submit_sm_resp". This field contains SMPP error code.
  • "smscErrorCode" - if message is rejected by the supplier using DLR, original value as sent by the supplier.

The client should reply with the HTTP status code 2xx (e.g. 200 or 202). The response body will be ignored.

7.4 Values of ErrorCode Field

Values sent in errorCode field can be found in the document HTTP Bulk Service API in 2.5.1 DLR Error Codes section. These are unified values of the SMS platform.
NOTE: For unified values to work, Error code mapping must be defined on the gate connections.

7.5 Values of SMSCErrorCodeNack Field

These values can be found in SMPP Protocol Specification 3.4 issue 1.2 document, section '5.1.3 command_status'. Many supplier systems extend this error table with custom errors that can be found in Supplier specification.

7.6 Values of SMSCErrorCode Field

Values of this field can be obtained from the gate supplier. These values are sent by the supplier using DLR, but they are not standardized, and each supplier is using own table.

Often is the case that suppliers do not provide table they are using, unless asked specifically.

7.7 Mapping SMSCErrorCode to ErrorCode Field

SMS Platform allows Error mapping tables assigned to gate connections, and when configured - values of smscErrorCode are mapped to standard set in errorCode. Both fields are present in the DLR.

7.8 Values of DeliveryStatus and DeliveryStatusCode

These values are documented in SMPP Protocol Specification 3.4 issue 1.2 document, section '5.3.2.35 message_state', i.e. 5.2.28 section.

deliveryStatusCode DeliveryStatus
1 enroute
2 delivered
3 expired
4 deleted
5 undeliverable
6 accepted
7 unknown
8 rejected

8. Sending MO Messages

MO notifications are sent to the URL provided by the customer and are being set up in the system. This URL must be either HTTP or HTTPS, with POST method, accepting content type 'application/json' and body payload encoded as JSON with UTF-8 encoding.

Example of an MO notification payload:

{
   "msgId": "8b6792bb-4014-b000-800c-80626935f2a6",
   "accountName": "200114_HTTP_Test",
   "smppServiceType": "",
   "smppSourceAddrTon": 1,
   "smppSourceAddrNpi": 1,
   "smppSourceAddr": "4915173185565",
   "smppDestAddrTon": 2,
   "smppDestAddrNpi": 1,
   "smppDestinationAddr": "72589",
   "smppEsmClass": 0,
   "smppProtocolId": 0,
   "smppPriorityFlag": 0,
   "smppScheduleDeliveryTime": "",
   "smppValidityPeriod": "",
   "smppRegisteredDelivery": 0,
   "smppReplaceIfPresentFlag": 0,
   "smppDataCoding": 4,
   "smppSmDefaultMsgId": 0,
   "smppSmLength": 140,
   "smppShortMessageBase64": "ABjnQECAGSik9/ujbIAAQFZK+YzT1Ru7Lqppw/0lDBlH1G3O7yBVVSgwoUAAAh0BCrAIygCAAFgkL0eRwpuMs4s/7GLEKU17vG6LH8tIUEY42weMxz7pc9KHFVYEcAO2H/81yIvUMVjR5hHJo45dlbBYV03PEp54t07KrNDkR7FWMv9/ob09GTE3eEA=",
    "tlv": [...]
}

The field "msgId" is generated by the API and used for references when troubleshooting. Except for smppShortMessage, encoded using Base64 encoding, other fields are sent either as an integer or string.

The client should reply with the status code 2xx (e.g. 200 or 202). The response body will be ignored.

MO messages are always sent with all the received fields included. This is the safest way to enable customers to decode various fields that can be sent in MO.

MO messages are received from the supplier typically as deliver_sm PDU, and there may also be TLVs included. These fields (including TLVs) are sent to the customer without change. TLV section is optional. If no TLV fields are present, the section will be omitted.

8.1 Long MO Messages

When long MO messages are submitted over the API - they will be submitted as they are received from the supplier link. The most common case is that long MO messages are received as split into concatenated parts, so they will be submitted to a Binary-safe HTTP client as they are received, in parts.

9. References

  1. HTTP Bulk Service API, version 4.1.0
  2. SMPP specifications v3.3, v3.4 and v5.0, please find them on smpp.org page.