MOBITEK SMS Engine — Enterprise Edition Manual

REVISION HISTORY

COPYRIGHT

Copyright © 2015 – 2019 MOBITEK System Sdn. Bhd. All rights reserved.

No part of this document may be reproduced, distributed, stored in a retrieval system or translated into any language, in any form or by any means, electronic, mechanical, magnetic, optical, photocopying, manual or otherwise, without the prior written permission of MOBITEK System Sdn. Bhd.

TRADEMARKS

MOBITEK^® is a registered trademark owns by MOBITEK System Sdn. Bhd.
Product names, logos, brands and other trademarks referred in this document are the property of their respective trademark holders and are used only to directly describe the products being provided.

DISCLAIMER

MOBITEK makes no representations or warranties with respect to the contents hereof and specifically disclaims any implied warranties of merchantability or fitness for any particular purpose.

Furthermore, MOBITEK reserves the right to revise this publication and to make changes from time to time in the contents hereof without obligation to notify any person of such revision or changes.

ACKNOWLEDGEMENT

We like to extend our appreciation to the following persons for their contribution in revising this manual:-

• Mr. Muhamad Faezudin bin Hairuddin, UNIVERSITY SAINS MALAYSIA
• Ms. Nur Shafiqah binti Muhamad Baharum, UNIVERSITI PERTAHANAN NASIONAL MALAYSIA

INTRODUCTION

MOBITEK^® SMS Engine – Enterprise Edition is a Windows Service that sends and receives SMS. System integrator and software developer (SI/SD) do not need to write any code to build a SMS Gateway. The MOBITEK^® SMS Engine – Enterprise Edition uses MS SQL Server database to store incoming SMS, and to check if there is any pending SMS to be send out.
SI/SD only needs to insert record into “outbox” table to send out SMS, and retrieve record from “inbox” table to read SMS. The MOBITEK^® SMS Engine – Enterprise Edition has taken care all SMS transactions.

FEATURES

1. Release – version 6.2
2. Running as Windows Service.
3. New: manual routing
4. New: concatenate multiple SMS, i.e. support sending SMS longer than 160 characters and joining them into 1 SMS
5. New: support Oracle 11g (not in standard package, separate license is required)
6. MOBITEK^® SMS Engine – Enterprise Edition will always try to initialize connection with modem, e.g. If PC/server is rebooted, the service will automatically establish connection with GSM modem.
7. Auto-reconnection with modem – if modem is disconnected from PC/server (for whatever reasons), and then re-connected at later stage, service will automatically establish connection with modem.
8. Works in Windows Server 2003 / 2008 / 2012 32 bit and 64 bit.
9. Uses Microsoft SQL Server, MySQL Server or Oracle 11g to store incoming SMS, and to process outgoing SMS in database. Support both Windows and SQL authentication mode.
10. Prevention of any lost of SMS:–
    1. During initialization, if there are any database error, it will not send nor read SMS, therefore no message will be lost.
    2. Auto-reconnection with database server.
11. Sending of SMS in alphanumeric character (text) and in Chinese character (Unicode).
12. Intelligent SMS Sending Logic:-
    1. Scheduling of outgoing SMS – based on the date and time set in “datetime_schedule”
    2. Prioritization of outgoing SMS – earliest “datetime_schedule” will be send out, and all “datetime_schedule = null” will be last to send out
    3. FIFO – if there is neither scheduling nor prioritization, then “FIFO” rule will be applied – SMS Engine will process the 1st record in the outbox until the last record.
13. Configure the direction of individual modem:-
    1. 2 way send and read SMS;
    2. 1 way read SMS; or
    3. 1 way send SMS.
14. Scalable Engine – support 1 or more MOBITEK^®MobiGATE^TM  without any need to reinstall Engine.
15. Load balancing – 4 modems work together to broadcast message. It is NOT on “round-robin” basis.
16. Fail-over – if modem no.1 cannot send out the SMS, modem no. 2 will take over, and so on until modem no. 4.
17. Dynamic queue size – the Engine will dynamically allocate queue size, if more modems are successfully initialized then queue size will be increased.
18. Configurable maximum queue size – the maximum queue size can be changed, default size is 8 SMS/records per modem.
19. Routing – sophisticated and intelligent routing logic that can be easily configured:-
    1. Modem will send to designated prefix mobile number. Good for cost saving as SMS sent within the intra-network is cheaper than inter-network.
    2. Modem will be or not be used.
20. Number of retries – can be configured. If SMS fails to be send, it will retry until the specified time.

LIMITATION

1. Sending of SMS in alphanumeric character with maximum of 160 characters length.
2. Sending of SMS in Chinese character with maximum of 70 characters length.
3. Does not support receiving concatenate or long SMS. Receiving of SMS is limited to 160 characters (ASCII) or 70 characters (Unicode) per record in the “inbox” table; if exceeded then the incoming SMS will be stored in multiple records in the “inbox” table.

REQUIREMENT

1. System integrators and software developers must posses knowledge of:
   1. SQL Database Administration for:-
      • Microsoft SQL Server, Microsoft SQL Server Enterprise Manager, Microsoft SQL Server Query Analyzer;
      • MySQL Database Server; or
      • Oracle Database Server
   2. Database programming:-
      • ODBC, ADO, ADO.net; and
      • SQL statement
   3. Windows Service Administration
2. MOBITEK^®MobiGATE^TM
3. SIM card(s)
4. Supported operating system: –
   1. Windows Server 2003 / 2008 / 2012 / 2016 Server 32-bit and 64-bit;
   2. Windows XP / Vista / 7 / 8 / 10 32-bit and 64-bit.
5. Microsoft .NET Framework Version 2.0
6. Microsoft .NET Framework Version 4.0 if using Oracle 11g
7. Support SQL Database Server:-
   1. Microsoft SQL Server 2000 / 2003 / 2008 Enterprise Edition;
   2. Microsoft SQL Server 2005 Express Edition;
   3. MySQL version 4.1 and above with MySQL ODBC driver version 5.1 in Windows OS;
   4. MySQL version 4.1 and above with MySQL ODBC driver version 5.1 in Linux OS; or
   5. Oracle 11g (separate license required)

 SYSTEM ARCHITECTURE

INSTALLATION of SMS ENGINE ENTERPRISE EDITION

1. Install Microsoft .NET Framework Version 2.0
   • install it from the USB thumb drive; or
   • download from (http://www.microsoft.com/downloads/details.aspx?familyid=0856eacb-4362-4b0d-8edd-aab15c5e04f5&displaylang=en)
2. Add a new database or use existing database.
3. Execute SQL script (in USB thumb driveCD) to add new tables to a new database or existing database.
4. For MS SQL database server, the sql script is “mssql-smsee6.sql”;
5. For MySQL database server, the sql script is “mysql-smsee6.sql”.
6. The SQL script will generate 4 tables:
   1. inbox
   2. outbox
   3. modem_state
   4. retry_send

      Note: Please make sure you have back-up your database before executing “smsengine.sql”. We will not be held responsible for any data loss nor data corruption.

7. Find out the COM port number of each modems of MOBITEK® MobiGATE^TM .
8. Install MOBITEK® SMS Engine by running the file “Setup_SMSEngine6.exe”. Go to the SMS Engine installation folder located at C:\Program Files (x86)\SMS Engine Enterprise Edition right-click on the folder and select “Properties”. Click on “Security” tab -> “Edit”.
   
9. Select “Users” and check for “Full Control”. Click “Apply”.
   
10. Using Notepad, edit the configuration file “config.xml”, located at C:\Program Files (x86)\SMS Engine Enterprise Edition. The contents of config.xml is as follows:
    

    <SMSConfig> <RETRY_SEND>3</RETRY_SEND> <QUEUE_SIZE>16</QUEUE_SIZE><ROUTING>P</ROUTING><MODEM_LIST> <MODEM><ID>1</ID> <PORT>41</PORT><ENABLED>true</ENABLED> <MODE>B</MODE> <TARGET_PREFIX>*</TARGET_PREFIX></MODEM><MODEM> <ID>2</ID> <PORT>42</PORT><ENABLED>true</ENABLED> <MODE>B</MODE> <TARGET_PREFIX>*</TARGET_PREFIX></MODEM><MODEM> <ID>3</ID> <PORT>43</PORT><ENABLED>true</ENABLED> <MODE>B</MODE> <TARGET_PREFIX>*</TARGET_PREFIX></MODEM><MODEM> <ID>4</ID> <PORT>44</PORT><ENABLED>true</ENABLED><MODE>B</MODE> <TARGET_PREFIX>*</TARGET_PREFIX> </MODEM> </MODEM_LIST><DATASOURCE> <DBTYPE>MySQL</DBTYPE> <SERVER>localhost</SERVER> <USERNAME>root</USERNAME> <PASSWORD>root</PASSWORD> <DATABASENAME>smseee6</DATABASENAME> <AUTHMODE>SQL</AUTHMODE> </DATASOURCE></SMSConfig>

    This table explains the node in the “config.xml” file:

    NODE	VALUE	DESCRIPTION
    <RETRY_SEND>	A number, starting from 0 <RETRY_SEND>0</RETRY_SEND> = no retry <RETRY_SEND>3</RETRY_SEND> = retry 3 times	Enter the number of retry in the event if SMS cannot be sent. The total number of sending is 1 + number of retry. This is a global setting that will affect all modems.
    <QUEUE_SIZE>	A number, default is 16. <QUEUE_SIZE>16</QUEUE_SIZE> = Engine will select 16 records from outbox table Minimum is 16. Maximum is unlimited	Maximum number of records that the Engine will select from the outbox table. The larger the queue size, the lesser connection/query is made to the SQL Database.
    <ROUTING>	Value can be “A”, “P” or “M”. <ROUTING>A</ROUTING> = automatic routing and load balancing <ROUTING>P</ROUTING> = routing by pre-fix based on value in <TARGET_PREFIX> <ROUTING>M</ROUTING> = manual routing based on value in the outbox.target_modem	To set the routing mode. This is a global setting that will affect all modems.
    <MODEM>	N/A	Setting for each modem. If there are 8 modems in MobiGATE, then there must be 8 <MODEM> nodes. If there are 4 modems, then there must be 4 <MODEM> nodes.
    <ID>	A number starting from 1. <ID>1</ID> = this modem’s ID or number is 1 <ID>2</ID> = this modem’s ID or number is 2	ID of modem or modem number. Depending on the model of MobiGATE, the range of ID can be from 1 to 4 or 1 to 8.
    <PORT>	A number, starting from 1 <PORT>9</PORT> = this modem is connected to COM port number 9	COM port of which the modem is connected. Please refer to “Device Manager > Ports” to find out the COM port numbers.
    <ENABLED>	Value is either “True” or “False”. <ENABLED>True</ENABLED> = modem is enabled <ENABLED>False</ENABLED> = modem is disabled, neither SMS will be send nor read on this modem	To enable or disable a modem. Engine will need to be restarted to take effect.
    <MODE>	Value can be “B”, “R” or “S”. <MODE>B</MODE> = bi-directional, modem will read and send SMS <MODE>R</MODE> = one way read SMS only <MODE>S</MODE> = one way send SMS only	To set the direction mode of modem.
    <TARGET_PREFIX>	A number List of numbers, separated by comma blank “*” wild card <TARGET_PREFIX>016</TARGET_PREFIX> = this modem will only send out SMS to mobile numbers with 016 prefix <TARGET_PREFIX>017,012</TARGET_PREFIX> = this modem will only send out SMS to numbers with 017 and 012 prefix <TARGET_PREFIX></TARGET_PREFIX> = this modem will NOT send out any SMS <TARGET_PREFIX>*</TARGET_PREFIX> = this modem will send SMS to any mobile numbers	To configure the prefix rule, modem will only send out SMS if prefix rule is matched. The routing must be set to <ROUTING>P</ROUTING> in order for the target prefix to work, else it will be ignored
    <MODEM> <ID>1</ID> <PORT>41</PORT> <ENABLED>true</ENABLED> <MODE>B</MODE> <TARGET_PREFIX>17,012</TARGET_PREFIX> </MODEM>	N/A	Modem ID 1 is connected to COM port number 41. It is enabled, and the mode is send and read SMS. It will only send SMS to numbers with 017 and 012 prefix.
    <DATASOURCE>	N/A	Settings for SQL Database Server
    <DBTYPE>	“MYSQL” or “MSSQL” <DBTYPE>MYSQL</DBTYPE> = Database server is MySQL <DBTYPE>MSSQL</DBTYPE> = Database server is MS SQL	To set which type of database server is used.
    <SERVER>	“IP address” or “name of server” <SERVER>192.168.1.66</SERVER> = IP address of the database server <SERVER>localhost</SERVER> = database server is on local server <SERVER>127.0.0.1SQLEXPRESS </SERVER> = IP address of MS SQL Express database server (note: for MS SQL Express Edition, add a suffix – “SQLEXPRESS” , e.g. 127.0.0.1SQLEXPRESS)	To set the IP address or name of database server.
    <USERNAME>	<USERNAME>root</USERNAME> = user name is “root”; default for MySQL <USERNAME>sa</USERNAME> = user name is “sa”; default for MS SQL	To set the user name of the login account of the database server
    <PASSWORD>	<PASSWORD>root</PASSWORD> = password is “root”; default for MySQL <PASSWORD>sa</PASSWORD> = password is “sa”; default for MS SQL	To set the password of the login account of the database server
    <DATABASENAME>	Alpanumeric. <DATABASENAME>smseee6</DATABASENAME> = name of the database used is “smseee6”	To set the database name used.
    <AUTHMODE>	Value can be “SQL”, “WIN” or blank <AUTHMODE>SQL</AUTHMODE> = MS SQL Server will use SQL authentication mode <AUTHMODE>WIN</AUTHMODE> = MS SQL Server will use Windows authentication mode <AUTHMODE></AUTHMODE> = leave blank if using MySQL Database Server	To set the authentication mode, only applies for MS SQL Database Server. For MySQL, leave it blank.

11. Save “config.xml” after all changes are made.
12. Make sure MOBITEK® MobiGATE^TM is powered on.
13. Start the service.
    
    1. Go to “Administrative Tools > Services”, look for service called “SMS Engine Enterprise Edition”, and click start.
    2. At later stage, you can configure the service to start automatically at “Start-up type: Automatic”.
14. Open log file to see if service is successfully started.
    1. Goto “C:Program FilesSMS Engine Enterprise Editionlog”, log file is saved as “yyyy_mm_dd.log”.
    2. Open current log file. If service is successfully started, then you will see this message:

18:44:59 SMS Engine Enterprise Edition service (version 6.0.0) started 18:45:01 Modem ID 1 : initialized

 

OUTBOX TABLE – SEND SMS

To send out SMS in text mode, the SQL statement is:

Insert into Outbox (message, destination) values ('Hello!', '+60172233111')

To send out SMS having Chinese character, the SQL statement is:

Insert into Outbox (message, destination, msg_type) values ('你好', '+60172233111', 'U')

To send out SMS by manual routing via modem ID 4 (provided that <ROUTING>M</ROUTING> is set), the SQL statement is:

Insert into Outbox (message, destination, target_modem) values ('Hello!', '+60172233111', '4')

To schedule a SMS to be send out at a predetermined date, the SQL statement is:

Insert into Outbox (message, destination, datetime_schedule) VALUES ('Happy New Year!', '+60176096718', '2020-01-01 00:01:00');

SMS Engine implements FIFO (first in first out) rule for processing messages to be send out. That is the 1st message that is inserted into the table will be the 1st message to be send out.

INBOX TABLE – READ SMS

SQL Statement:

• To read all new SMS

Select * from Inbox where read_status = 'N'

• To change the status of new SMS to “read”

Update Inbox Set read_status = 'Read' Where Status = 'N'

or

Update Inbox Set read_status = 'Read' Where msg_id = 'ID of the message that SI/SD is processing'

RETRY SEND TABLE

This table is used for logging number of retries for a message.

For example, if a record with ID “118” has this value:-

It means – modem with ID number 1 had retried 3 times to send out the message having ID “A3210”.

FREQUENTLY ASKED QUESTIONS

Last updated on 22nd of July, 2015
Q: I am using U Mobile post-paid, why does the Engine’s log shows “not able to connect to GSM network” and the red LED on modem is on but not blinking?
A: U Mobile post-paid is 3G only. Therefore, the modem is unable to connect to GSM network using a 3G SIM card. Please request a 2G SIM card from U Mobile.

Q: We are facing issue – when sending the symbol “~”, the SMS received on hand phone shows “ű” or a “square” or a “?”, what is wrong?
A: These characters are NOT the default alphabet defined by the GSM standard:-

{}[]~| and the Euro symbol: €

If you want to use the above character/alphabet, then you need to set “msg_type=U” and limit your maximum number of character per SMS to 70.

Q: The light of the first modem is not blinking but constantly on, while the rest are blinking.
A: Do the following:-

1. Make sure the SIM lock is disabled. Use your hand phone to disable the SIM lock function.
2. Remove SIM cards, use a rubber or eraser, gently rub the gold area of the SIM card to clean it.
3. The SIM card may be defective exchange for another one.
4. Check your balance or expiry date if you are using pre-paid.

Q: I am using SIM cards having prepaid, how do I check my balance without stopping the service?
A: You can goto the web site of the GSM network operator (TELCO/CELCO/carrier) and check the balance on-line. Here are a list of the web sites for each of the CELCO:-

Hotlink	http://www.hotlink.com.my/
Digi	https://www.digi.com.my/ocs/myaccount/main/home.do
Xpax	http://www.xpax.com.my/index.php
U Mobile	http://www.u.com.my

Q: I am using SIM cards having prepaid, I want to reload or top-up my account, how do I do so without stopping the service or removing the SIM card?
A: You can goto the web site of the GSM network operator (TELCO/CELCO/carrier).

Q: The log file is good for trouble shooting. Any setting for us to control the size of the file?
A: It is good to keep log for audit and trouble shooting purposes. And nowadays, server has large hard disk capacity, so the log file hogging space is not an issue.

Q: The service will always try to initialize with MOBITEK^® MobiGATETM. What about database connection? What if the service started before SQL Server started? Will the service try to re-connect with SQL server?
A: Yes, it will. When the SMS Engine Enterprise Edition service is started, it will always try to initialize the GSM modem, and will always try to connect to SQL server.
In order that the service is not stopped (for whatever reasons), goto “Administrative Tools > Services”, right click on “SMS Engine > Properties > Recovery tab”.
Under “First failure:” select “Restart service”
Under “Second failure:” select “Restart service”
Under “Subsequent failures:” select “Restart service”
This will make sure that the SMS Engine – Enterprise Edition service is always running, and it will always attempt to initialise the GSM modem and to connect to the database.
However, if your SQL Server fails, then not only SMS Engine – Enterprise Edition will not run properly, other applications are affected as well. Therefore, it is your responsibility to ensure that SQL Server does not fail.

Q: Can I modify the structure database or tables?
A: It is important that the database schema or structure of 3 tables (inbox, outbox, modem_state, retry_send) are intact. You must not modify the fields nor data type. Otherwise the SMS Engine will fail to operate properly.
You must use the SQL script provided to create 3 tables, and you must back-up your database before you do so. We do not assume any liability if there are any data corruption or data loss.

Q: How does the SMS Engine process each record in outbox table in order to send out SMS?
A: The process flow is as follows
P > L > S
or
P > L > R > S
or
P > L > R > F
The final value is either ‘S’ or ‘F’.
Note: System Integrator must not enter any values into “sent_status” field, doing so will disrupt the flow or will cause SMS Engine to process the record improperly.

Q: Do I need to enter “P” in “sent_status” field?
A: You do not need to enter the value ‘P’ for “sent_status”, it is automatically inserted. The SMS Engine will automatically update the “sent_status” field.

Q: May we know how to sequence the SMS Engine process the records in “outbox” table? In other words, which message is sent out first.
A: The rule for processing record in “outbox” table is FIFO. The first record inserted into “outbox” table will be the 1st record sent out.

Q: What is the delay for next retry?
A: There is a delay of 2 seconds.
Note: if you have 3 records in outbox, if the 1st record fail to send out, it will retry until the number of times stated in 3 before moving to 2nd record, will do the same, before moving to 3rd record.

Q: What is the purpose of “modem_state” table?
A: You must not touch the “modem_state” table, it is irrelevant to your application. It used by the SMS Engine Monitor for reporting purpose.

Q: Can we have a centralised database if we install the SMS Engine in a 2×2 configuration?
A: The current SMS Engine – Enterprise Edition does not support centralised database. I.e. if 2 or more Engines are installed on different servers, then each Engine must have its own database.

Nevertheless, if you need a “centralised database mode”, we are able to develop a special edition with a centralised queue manager. Please let us know, we will quote the license fee to you.

An alternative solution is that your application will manage the queue — you will decide whether to insert record into “database 1” or “database 2”.

TIPS

These are import factors to ensure SMS Engine operates smoothly:

1. config.xml properly configured
2. Choose the correct SQL script to be executed, and 3 tables (inbox, outbox, modem_state) are created or added to your preferred database name
3. SMS Engine service is started
4. Other factors which are beyond are our control that will also affect SMS Engine are:
   1. whether SQL Database Server is running
   2. whether LAN is running, and firewall is opened for SMS Engine (if database server and SMS Engine are on separate machines)
   3. whether there is database connection
   4. whether PC / server / Windows is running
   5. whether GSM network is running
   6. whether SIM card is barred
   7. whether SIM card has money

TERMS AND CONDITIONS ON SUPPORT

Please write to support@mobitek.my with:

1. company name
2. invoice number (located at back of modem)
3. description of problem
4. version number of the SMS Engine – Enterprise Edition (right click on “C:Program FilesSMS Engine Enterprise EditionSMSSvc.exe”, select “Properties > Version tab”)
5. attach screen capture of “VCOM” showing all COM ports and its IP address
6. attach log file (located at “C:Program FilesSMS Engine Enterprise Editionlog”)
7. attach “config.xml” file

APPENDIX 1: RELEASE NOTES

APPENDIX 2: GUIDE TO ADD 1 OR MORE SMS MODEM TO EXISTING SMS ENGINE

(removed)

APPENDIX 3: GUIDE TO INCREASE MODEM ACCESS LICENSE FROM 4 TO 8

(removed)

APPENDIX 4: HOW TO SET-UP MSSQL SERVER EXPRESS

1. Install Microsoft SQL Server 2008 R2, you can get it from Microsoft official site.
2. Run Microsoft SQL Server Management Studio.
3. On the first run, “Connect to Server” will pop-up
   

   1. Choose:-

      • Server type : Database Engine
        
      • Server name: <browse for more → local server → database engine →your pc name >
        
      • Authentication : Windows Authentication
        

      • Press connect.

4. On “object explorer” → right click on databases → create new database.
   
   
   
5. Give it a name “smseee” and press OK
6. On “object explorer” → right click on your new created database → New Query
   
   
7. <plang=”en-US”>Copy and paste the sql command from “mssql.sql”

• Execute the command

9. The database is set up, in case you want to know about the detail of you database for further use in MOBITEK^® SMS Engine (Enterprise Edition), right click on your database → properties
   
   
10. In “config.xml” file, edit as follows:-
    
    

    <DATASOURCE><DBTYPE>MSSQL</DBTYPE><SERVER>DELL-D630SQLEXPRESS</SERVER><USERNAME>abc</USERNAME><PASSWORD>abc</PASSWORD><DATABASENAME>smseee5</DATABASENAME><AUTHMODE>WIN</AUTHMODE> </DATASOURCE>

APPENDIX 5: HOW TO SET-UP MySQL DATABASE

1. Run the “WampServer64” to install Apache, MySQL Database and PHP.

2. After successful installation, at the taskbar, click “Start All Services”.

3. Click “Localhost”.

4. Your localhost should be opened at the browser like below. Click at the “phpmyadmin”.
   

5. You will be directed to “localhost/phpmyadmin”. Click “New” on the left side.
   

6. Enter the nameas “smseee6” then click “Create”.
   

7. Click the “smsee6”.
   

8. Click at the ‘SQL’ one you are directed like below.
   

9. Go to your folder and open up your mysql-smsee6. Right click and click edit with “Notepad++” or “Notepad”.
   

10. Select all and copy the SQL script.
    

11. Go back to your “phpmyadmin” page. Right click and paste it at “Run SQL query/queries on database smseee6”. Then click “Go” button.
    

12. If successful, it will display the following result.:
    

13. There you can see 4 tables inside your database which are “inbox”, “modem_state”, “outbox” and “retry_send”.