Assignment 6: Protocols, or What did you say?

Click here to access the repository for this assignment. You will also need to copy your completed SerialComm.java file from Studio 6 into the communication package in order to get this assignment to work.

The idea

• The idea
• The background
  • Protocols
  • Our protocol
    • Header
    • Payload
• The assignment
  • Guidelines
• The check-in
  • The rubric

When two computers send data from one to another, it is the job of a protocol to specify how bytes transmitted by the sender are to be interpreted by the receiver. The idea is simple: if we specify very, very concretely how we expect our data, it becomes easy to communicate. Despite the vast differences between our Arduino and Java programs, so long as we create and follow a well-defined protocol, the differences barely matter.

In this assignment, you will build a pair of programs—one Arduino, one Java—that harvest real-world data, transmit it, and then receive it on another processor. Though this assignment focuses mostly on the data transmission, this centralized design is very useful in many circumstances. WIFI networks, at least in the small scale, rely on this, where many devices connect to a centralized hub.

Using a provided protocol, you will design an Arduino program to send and a sister Java program to receive real-world measurements.

The background

Protocols

No matter what we want to communicate, someone, somewhere, somehow has to decide upon a way of communicating it. Someone created a protocol. This is clear when looking at something like language: it has rules, syntax, grammar, symbols, definitions, and a slew of implicit and explicit rules. Though not as murkily defined as language, all forms of computer communications have the same structure.

Because we could never express the definition of one accurately enough ourselves, from Wikipedia, protocols are “the rules that define the syntax, semantics and synchronization of communication and possible error recovery methods.” In a word: rules.

If we define every bit of our protocol precisely enough—from the size of ints to the encoding used for chars to the ordering and grouping of data—we can easily (and independently) write software to both produce and consume that data.

We detail this week’s protocol below.

Our protocol

We chose to design a protocol that centers around the sending of individual messages. Each message we send will have a 1 byte header and a variable length payload. The header is basically used for book-keeping and the payload is the actual information being sent.

In our case, the payload will send key-value pairs: a key indicating the meaning of our value (is it a temperature or a string?, etc.) and a value for that key. A value might be a raw temperature reading, a filtered temperature reading, a potentiometer reading, a timestamp, an error message, etc.

Note: In this assignment, we are using the terminology key-value pairs; however, another nomenclature that is also very common is to call them tag-value pairs. Don’t be surprized if you see these terms used for essentially the same purpose.

Header

As stated above, the message header helps the communication protocol do some book-keeping. In our case, the header is designed to help us sync up the beginning of a message both at the sender and the receiver.

Consider what happens if the sender is delivering a series of two-byte integers, high byte first then low byte second, and the receiver somehow misses one byte. If this happens, all the integers that follow will be interpreted incorrectly by the receiver.

We will significantly decrease (but not completely eliminate) the possibility of this happening by starting each message with a magic number, a constant value that is known to both sender and receiver as always present at the start of a message.

When the sender wishes to transmit a message, it must start the message with the magic number. For our protocol, the magic number is 0x21, or an ASCII '!'.

When the receiver is ready to interpret an incoming stream of bytes, it knows that the first byte of a valid message will be the magic number. When it reads a byte from the stream, if the byte is not the magic number, it can discard that byte (and all subsequent bytes that are not the magic number) until it reads a byte that is the magic number. In this way, it is reasonably assured to be aligned with the beginning of a message.

Payload

The payload in our protocol is comprised of a key-value pair, where the key defines both the meaning and the format of the subsequent value.

The key is a fixed-length field: one byte. The value is a variable-length field, the number of bytes depending upon a number of factors.

For this assignment, we define the following keys:

• 0x30 debugging string in UTF-8 format, maximum of 100 characters long
• 0x31 error string in UTF-8 format, maximum of 100 characters long
• 0x32 timestamp, 4-byte integer, milliseconds since reset
• 0x33 potentiometer reading, 2-byte integer A/D counts
• 0x34 raw (unconverted) temperature reading, 2-byte integer A/D counts

As the semester progresses, we might add to the above list, but we won’t change the meaning of these already-defined keys.

For string values in UTF-8 format, the first two bytes are a two-byte integer that gives the length (number of characters) of the string. This is then followed by the ASCII characters of the string. In our protocol, characters are restricted to the range 0x01 to 0x7f (i.e., NULL—0x00—is not allowed, nor are characters in the extended range of the UTF-8 standard¹). In Java, you will need to create a String from the input ASCII characters. Put the characters in a byte array, e.g., byte[] chars, and use the following form of the String constructor: String(chars, StandardCharsets.UTF_8).

For more information about how string types are represented in Arduino C, take a look at this video:

For multi-byte values that represent a single number (e.g., 2-byte integers, 4-byte integers), the order of the bytes is most significant byte first and least significant byte last. In Java, take a look at the optional parts of Studio 6, and do those exercises if you haven’t yet done them.

The assignment

Locate and open the MsgReceiver.java file in your repository. This is where most of your work will go on the Java side.

1. Author enough of an Arduino sketch sender.ino to send debug messages (i.e., send the magic number !, the key 0x30, and then a UTF-8 encoded string, prefixed with its length)

   Send a few debug messages, making sure you’re confident doing so. Can you write a function called sendDebug()? It can accept a string literal as a parameter by taking a char* as an argument.

   If the Serial Monitor isn’t helpful enough (it won’t be, because you are no longer just sending ASCII characters), alter the Java MsgReceiver’s run() method to continuously read bytes from the input port. Ensure debug is true in your SerialComm so input bytes are printed in hex form.

2. Author enough of the Java program (MsgReceiver’s run() method) that you can receive a debug message and reliably observe the debug string on the console window. Send a few more debug messages.
3. Set up a delta-time loop, running at about 1 Hz, that sends a message containing the millis() value used to control the loop (i.e., save the return value from millis() in a variable so that you can use it both for the if test in the delta-time code and send it to the PC. Make sure to use an unsigned long). This is the 0x32 message.

4. Alter your Java code to parse the message and print it out nicely every time it’s received. Include the type of message (e.g., debug string, error string, potentiometer value, raw temperature value) as well as the value (the string value or integer value).

   Structure your receiver program as a finite state machine. We recommend reading our Introduction to FSMs guide, as it will make your program significantly easier to reason about.

   Messages that do not conform to the protocol should generate an error printed to the console that is visually distinct from the protocol-supported error message. Indent it and use a different format. I like !!!!! Error but that may be a little dire for your tastes.

5. Attach the potentiometer and temperature sensors to two distinct analog input pins. The wiring is the same as the last two assignments.
6. Update your delta time loop to also read the value of the potentiometer, sending it after the millis() message. This is the 0x33 message.
7. Likewise update your Java code to read this number and print it out nicely.

8. Continue doing this for the other keys: send and a receive a raw A/D counts reading from the temperature sensor. This is the 0x34 message.

   This hardware setup is the same as for Assignment 3; however, you will need to use analogReference(DEFAULT) to maintain compatibility with the potentiometer.

9. When the potentiometer reading is above a threshold value (feel free to use whatever value you chose in that assignment, or pick a new value) send a message with the error string “High alarm”. This key = 0x31 message should come at the end of all output.
10. On the Java side, when you receive a temperature value you should convert it to engineering units (degrees Celcius), filter it (using a rolling average filter like you used on assignment 3), then display the raw, converted, and filtered temperature values. Note, your conversion code is now in Java, not in C. Also, the math isn’t the same either, because the analog reference voltage is now 5V instead of the previous 1.1V. Because the 5V reference isn’t as precise, don’t worry if your temperature values are a few degrees off.

Guidelines

1. Try to develop an intuition for the raw numbers behind ASCII. For example, what ASCII characters do our protocol’s keys correspond to?

   This will be helpful if your Java program is behaving strangely and you need to look at Arduino Serial Monitor. You should be able to get a sense of the messages as they come through: you’ll see the ! as the magic number, then the ASCII character of the key, then “gibberish.” Understanding which numbers go to which ASCII characters will help you make sense of that gibberish.

2. Make sure your program generates errors if it receives input it didn’t expect. While silently failing is sometimes good (see Defensive Programming), in your case it will make it harder to understand where your program is going wrong.

   Be loud! Print error messages! User error is the best type of error because it’s not your fault.

3. Read your analog values, parse them, filter your data, and then print all your messages. It will make it easier for us to grade your assignment (and also easier for you to debug).

4. Send debug strings a lot. Since they are the first thing you implement, they are also the first thing you can be confident works, so if something is broken, falling back to them may be just what you need.

   Use them only for debugging and small program notes, though. They’re “debug strings” for a reason.

The check-in

1. Commit all your code (make sure to add any new files to your repo first).
2. Check out with a TA.

The rubric

• 100pts total:
  • Messages adhere to the protocol described in class (20 pts)
  • Messages are interpreted correctly by the Java program (20 pts)
  • Message types supported (25 pts)
    • Debugging string in UTF-8 format (5 pts)
    • Error string in UTF-8 format (5 pts)
    • Timestamp represents the number of milliseconds since reset, 4-byte integer (5 pts)
    • Potentiometer reading in A/D counts, 2-byte integer (5 pts)
    • Raw (unfiltered) temperature reading in A/D counts, 2-byte integer (5 pts)
  • Proper conversion and filtering of temperature on Java side (10 pts)
  • SerialComm debug showing received bytes (10 pts)
  • Correct wiring and sensor readings (15 pts)

As usual, -1 points if you do not commit to Github before demoing.