Real-Time Linux State Machine
- 1 State Machine Basics
- 2 Practicalities: Connecting to the State Machine
- 3 Probabilistic Jumps
- 4 Exercises
- 5 Getting information back from the RTLSM : Getting events and the disassembler
- 6 Previous: General overview
- 7 Next: RTLSM and trial structure
State Machine Basics
States, simple inputs, simple outputs, states' internal clocks
As the name indicates, a state machine is composed of a series of states. External or internal inputs (e.g., pokes, levers, etc) may cause transitions between these states; and each state may provide some output signals to the world (e.g., controls for lights, sounds, water).
Self-timers. In the RTLSM, each state is also associated with an internal clock, the self_timer, which will run for a certain time duration. When you define a state, you can also define how long that self-timer will run for. Whenever you transition into a state from a different state, the post-transition state's self-timer gets started. When it reaches the specified time, the self-timer generates an internal event called Tup. You can choose to either use or ignore this Tup event.
Events and state transitions. You can define state transitions that follow from events: for example, in the schematic at right, we've defined a state machine in which there is a state called "light_on" in which the external event "Cin" (poking into the center poke) causes a transition to the state "reward." The "Tup" event from the self-timer can be used the same way. In the example at right, if 2 seconds have elapsed since the machine entered "light_on," a Tup event will cause a transition to the "punish" state.
Outputs. Each state can also be associated with different outputs. For example, while in the state "light_on," the center poke light will be turned on. Outputs can either be constant (as in the case of digital outputs like lights and water valves), meaning the outputs stay on the whole time the machine is in the state that turned them on; or they can be triggers that occur instantaneously upon entering a state (as in the case of triggering sounds on/off).
With these elements, we built the state machine at right, that does the following: Like all RTLSM state machines, it starts at state 0. By convention, we don't use state 0, but jump as fast as we can to the start of our program (given the RTLSM's 6KHz cycle, that's 1/6th of a ms). Then, in "my_start" we wait for the subject to poke into the center poke, with no time limit for how long we wait. Poking into the center poke takes us to "light_on," where we turn the center light on. If the subject pokes again in the center within 2 secs after the light was turned on, they get a 200 ms water reward. If they don't poke in time, the light will turn off and they get a 1 sec white noise punishment. And then the whole cycle starts again. Here's the code for very simple state machine.
"Scheduled Waves" are like alarm clocks not attached to a state
Suppose you want the following behavior: when a subject pokes into the center poke, the center light goes on; if the subject moves out of the center poke, the light goes off; and this can happen no matter how much the subject pokes in or out. But 2 seconds after the subject poked into the center poke for the first time, the two side lights will go on, regardless of what is going on with the center poke, and the state machine will then stop. How can we achieve this? If all we have at our disposal is what we've decribed above for the RTLSM, we're going to have a hard time obtaining the desired behavior. The reason is that the self-timers are all attached to states, and we need different states for the light on/light off conditions. For example, if we change state (e.g., change from light on to turning the light off), we lose track of how long it was since the subject first poked into the center poke. Given enough states, we can write a state machine that will do it, but it'll be clumsy.
Scheduled Waves are a more elegant way to program behaviors like this. Essentially, they are timers that are not attached to any state. You define how much time they will count for, and you can trigger their start from any state in the same way that you can trigger the start of a sound (i.e., states can trigger Scheduled Waves as one of the states' outputs). When they reach their specified time, Scheduled Waves generate events, and you can react to those events just as you would to an external event, such as a Cin.
In the example to the right, we've defined a Scheduled Wave, "my_wave," to be 2 secs long. We start so that poking into the center takes us to "start_wave," which we use merely to trigger the start of "my_wave" and from which we jump immediately to "light_on". The two states "light_on" and "light_off" simply switch between each other upon pokes in and out, with "light_on" having the center light on. We also define both of these states so that if my_wave reaches its countdown (and generates the event "my_wave_In"), then there is a transition to state "side_lights." This way, it doesn't matter how long we stay in each of "light_on" or "light_off", nor does it matter how many times we switch between them: when my_wave goes "ping!", we transition to "side_lights." Here's the code for simple scheduled waves example.
Practicalities: Connecting to the State Machine
To make a connection to the State Machine from Matlab, you create a State Machine object, and then you Initialize it. State Machine objects are objects of class @RTLSM2, which on creation connect to an RT Linux RTLSM server, or the emulator.
To connect to an RTLSM Emulator running on the local machine: >> flush; newstartup; % Initialization scripts >> sm = RTLSM2('localhost'); OR >> sm = RTLSM2('127.0.0.1'); Alternatively, if you really want to connect to a physical rig, you connect to an @RTLSM2 object using the IP address of the actual RT Linux machine. You create it and give the creator an argument specifying the IP address or name of the RT Linux machine: >> sm = RTLSM2('myrtlinux.myinstitution.org'); After you've chosen one of the above to, call its Initialize.m method: >> sm = Initialize(sm);
Now that you've got your connection, what shall you do with it? Currently, the best place to start learning how to use the state machine is by looking at example code in the State Machine Assembler. Go to ExperPort/Modules/@StateMachineAssembler/Examples. Look at example1.m. Then do the exercises below.
>> sma = add_probabilistic_jump(sma, state_name, prob_vector, states_vector)
adds a state from which the state machine immediately jumps out with probabilities defined by prob_vector, to the states indicated in states_vector. This allows you to make stochastic choices from within a single state machine diagram. See documentation here .
State Transition Matrices
These excercises are best done in the context of using two pre-provided example m-files as the basis for them. These examples are found in
Modules/@StateMachineAssembler/Examples/example1.m and example2.m
State Machine Basics
(1) Write and test a state machine such that whenever you poke into one of the nose cones, the light in that nose cone turns on and stays on until you pull your finger out.
(2) Write and test a state machine such that whenever you poke into one of the nose cones, all lights turn on for two seconds, and a two-sec long 400 Hz pure tone is played. If you pull your finger out before the two seconds are up, the lights and the sound should turn off.
Using Scheduled Waves
(3) Write and test a state machine that turns on the leftmost light on and off on a 2 Hz cycle; and that simultaneously turns the center light on and off on a 3.9 Hz cycle. Try writing it without using Scheduled Waves; then try writing it using Scheduled Waves.
(4) Write and test a state machine such that if you poke into the center cone for a second or more, and then poke into either of the side cones, you will get 100 ms of water and 100 ms of light in the side cone you poked into. But here’s the interesting part: within that 1 sec in the center cone, you are allowed to pull out of it briefly: as long as you pull out of the center cone and return to it within 75 ms, your 1 sec will still accumulate and you will still get your reward in the side cone. Thus, for example, if you poke into the center cone for 500 ms, pull out and return 70 ms later, you will have to keep your finger in the center cone for another 430 ms before you can go to the side cone to get your reward. As above, for fun try writing this without Scheduled Waves.
(5) A difficult one, entitled “reward proportional to poke length:” write and test a state machine such that if you poke into center cone for T seconds, and then poke into a side cone, the light in the side cone you poked into turns on for T/2 seconds. The core state machine code has been rewritten to allow far more graceful handling of situations like this, but is still in the testing stages.
Getting information back from the RTLSM : Getting events and the disassembler
How does the Governing Machine get information from the State Machine? While the State Machine is running, the Governing Machine can get information by querying the state machine for a history of events are. In Matlab, this is done through two functions. Assuming that sm is a variable that represents a state machine object, the two functionss are:
>> nevents = GetEventCounter(sm);
>> events = GetEvents(sm, startevent_number, endevent_number);
The first function returns the number of events that have occurred since the State Machine was started. The second function returns a matrix that represents all the events that have occurred in between startevent_number and endevent_number, inclusive. “Events” are the occurrence of inputs, such as nose pokes, or timers going off, that have the capacity to cause state transitions. The events matrix has information on what those inputs are, on what state the State Machine was when those events happened, and on what time they happened. (Do “help GetEvents” for details on the return format of that function.)
See @StateMachineAssembler/Examples/example2.m for an example of using these functions.
You can also find example2.m here.
First, look through Modules/@StateMachineAssembler/Examples/example2.m to learn how the disassembler works. Then, write and test a state machine that does the following super-simple thing: it keeps track of how many Center, Left, and Right pokes there have been. On every new poke, it prints out the updated totals to the command window.