Editing GreenHouse Monitor Program

= What =
This project uses the Python Smart Terminal program to communicate with an arduino and monitor, record, and graph conditions in the green house. The main page for the application is at: '''[[Python Smart Terminal]]''' and it has a category page at '''[[https://opencircuits.com/index.php?title=Category:SmartTerminal Category:SmartTerminal]]''' ).   


This is really 2 programs: 1) an arduino program intended to monitor the environment in a green house and 2) its PC side ( often for a Raspberry Pi ) written in Python.  The Arduino has a simple command driven interface.  It is designed to run with [[Python Smart Terminal]] and an extension to it which is added when the terminal is set to the mode = GreenHouse.  Documentation on the construction of a Smart Terminal extension to support the green house monitor and save its data to a csv file are also part of this project and the documentation here.  Another adaptation for the SmartTerminal is described in [[SmartTerminal for Controlino]]

Note that since this was written the SmartTerminal has been updated and its operation and parameter set up have been upgraded and in some ways simplified.  I will be updating this documentation ( hopefully soon Feb 2018 -- did not hapen  ).  Another update the python side of the Green House Monitor used to be ext_process_gh.py but has now been absorbed into ( and is a mode ) in ext_process_env_monitor.py.  This documentation will perhaps be revised at some point to reflect that change.

This is an article started by Russ Hensel, see "http://www.opencircuits.com/index.php?title=Russ_hensel#About My Articles" '''About My Articles''' for a bit of info.

= Source Code =
Still working on where I will keep the source.  It is open source, email me in the meantime: [[User:Russ_hensel]]  The code is pretty straight forward, the heavy lifting is all left to the  [[Python Smart Terminal]].  There is nothing very fancy or special about the arduino program, get the source, read it.  Email me if there are issuses.  Also look at this wiki's category SmartTerminal (link at bottom of this page).

= Arduino GreenHouse Commands =
Commands are single letter strings, in some cases followed by numbers.  All end with a <cr>.  All envoke responses all of which also end with <cr>

{| class="wikitable"
|-valign="top"	
|Command Purpose
|Send
|Example Response
|Comment
<!-------------------------------->
|-valign="top"
|Get version of arduino software
|v
|GreenHouse Monitor v3 2017 01 24.01
|Of course exact string changes with version.  This string is used by the SmartTerminal to verify the arduino connection.
<!-------------------------------->
<!--------------------------------
|-valign="top"
|Aquire Data
|a
|ok
|Data is held in arduino memory, access with further commands.
<!-------------------------------->
|-valign="top"
|Aquire the data.
|a
|ok
|Data is buffered in the arduino
<!-------------------------------->
<!-------------------------------->
|-valign="top"
|Get the Temperature
|t
|33.0 60.0
|In this version there are 2 sensors, data separated by spaces.
<!-------------------------------->
|-valign="top"
|Get the Humidity
|h
|43.0 55.0
|In this version there are 2 sensors.
|}

= Smart Terminal Extension =

In general when you want to make an extension you should find one that is similar to the one you want, copy it, then tweak the code.  This description will not quite do that because the extension ( and arduino program ) are already written.

== Copy Some Code ==

All the expensions that are written are in modules with names of the form xxx_processing.  Look at the file headers .... and pick one to copy.  If you run this copy it should start up without too much trouble.  But how do you get the code to execute? Say your module ( file ) is my_processing.py and the class name ( which you will probably change when you make the copy ) is MyProcessing.  You do not run your module, you run SmartTerminal.  The trick is to first make some changes to the parameter file.  What you need is to make sure that at the end of creating a Parameter class the final setting reflect something like:

<pre>
     self.ext_processing_module      = "my_processing"
     self.ext_processing_class       = "MYProcessing"
</pre>

are the setting that are made last.  More documentation the on the parameters is found in [[Smart Terminal Parameter Examples]].

== SmartTerminal Code Explanation ==

I will explain the green house code, use this explanation to build your own code.  This code is gh_csv_processing.py The really fine details are in the code comments, this documentation is at a higher level.  To activate the code have your parameter file ( this is automatic in the intermediate file by setting mode = "GreenHouseCSV" )

<pre>
    self.ext_processing_module      = "gh_csv_processing"
    self.ext_processing_class       = "GHCSVProcessing"
</pre>

=== What Is the Point ===

It is always nice to know when code is being explained what the code is supposed to do.  So here it is.

The idea is that the arduino will be set up in an environment like a green house and its job is to measure and report on variables like temperature and humidity.  The job of the smart terminal is to control that data acquisition and log it to a csv file.

=== Details ===

*The smart terminal need not be used in its smart mode, all the dumb terminal stuff still works and let you issue commands to the arduino and inspect the results.  
*In addition the smart function that are added include:

** Automatic probe of comm ports to find the arduino.
** Opening the comm port and logging the data to a csv file.

Lets get an overview of the methods ( for the detail read the code, and to actually understand all this you will eventually need to '''read the code''' ):

* __init__()  builds the class, gathers references to other objects in the application, and finally defines some instance variables for data processing later
* add_gui()   builds gui components and returns a reference to the GUI components, for Tkinter this is a frame, for Kivy it will probably be a layout.

Now a complexity of the application is that it is running in 2 different threads.  When the application starts up it has a single thread that after a bit of time creates the gui and runs its main method.  I call that the GUI thread.  The GUI thread starts another thread, ( the class SmartTerminalHelper ) which has its own methods and attributes.  In a multi threaded application you have to be careful what calls what.  Calls between the threads are particularly tricky once both are running.  For the more difficult issues of this type communication between the threads is managed by pushing data into queues which are thread safe and allow the threads to communicate safely with each other.  All this is in the code.  The GHCSVProcessing class is a bit complicated since parts of it run in one thread, parts in the other.  How do you tell what is what.  There are two ways one is to reason it out, one is to read the comments, this discussion should help. Init is called from the GUI thread which is the one that is "in charge" of the application.  It also calls __add_gui__ which since it adds to the GUI will create objects that are also executed in the GUI thread.  All the functions in the class are commented so you know what class they should be called from ( run in )

==== Additional Functions that Run in the GIU ====
Both of these are call back function called from the GUI we have created in add_gui, so they run in the GUI thread.  However both want to call functions in the helper thread.  There code is very simple, each one pushes its request into a queue which is processed by the helper thread when it get to it.

*cb_find_and_monitor_arduino()
*cb_end_helper( )

==== Rest of Functions Run in Helper ====

*find_and_monitor_arduino( )  This is the most important function in this class. Typically it is an infinite loop unless it encounters an unrecoverable error, or the GUI asks it to stop. It has the following functions:

** find work through the comm ports to find ones that ope.
** validate the port
** ask the arduino for data and save the data to a csv file
** keep displaying the received data in the GUI
** watch out to see if the GUI wants to interrupt ( and stop ) this routine.

Supporting functions for the above are:

*set_time()              saves the time of data acquisition for saving to the csv file
*process_temp_line()     takes a string ( one line ) of temperature and parses out the numbers on it ( as floats ) for saving to the csv file
*process_humid_line()    like process_temp_line() but for humidity


== Deeper Dive into find_and_monitor_arduino( ) ==

The functions in the GUI thread, including this one, are responsible for the receive function of the terminal when they are active.  They also need to see if the GUI needs to interrupt them.  To do this they will often poll the com port and check the queue_to_helper.  Now most of the time they are more or less sleeping waiting for the next time to acquire data.  They should not use time.sleep() as that will stop their polling.  Instead they call: check_queue_for( a_time ) in the HelperThread instance which acts like sleep() + the required polling.  All the functions called here should call this method often to keep the receive and interrupt alive.  ( there is a method for the helper to pass the receive processing back to the GUI thread, see the code for the details )  Other than this pooling the coding here is pretty straight forward.

* to find the arduino there is a method in the HelperThread instance that takes care of it, a couple of parameters control the details.
* set_time() is called to record the time of data acquisition
* the string requesting data acquisition is sent to the arduino
* the string requesting the temperature data is sent to the arduino, the received string is parsed by process_temp_line()
* the same process is repeated for the humidity.
* a function is called in the HelperThread instance to save the data to the csv file ( controlled in part by the parameters ).
* the function sleeps until it is time to repeat the cycle.

== Interrupt and Error Management in the Helper Thread ==

This is proving to be one of the more difficult aspects of the program, an I am still working on it.  I have however decided upon an approach which seems promising.  The plan, and what is implemented so far is to use various exceptions ( or exceptions with instance variables ) to unwind the code back to some recovery point.

The code for interrupting the helper thread is currently working as intended.  When the queue_to_helper is detected as being non empty during a helper function ( except the main polling one ) an exception is thrown which is caught back in HelperThread.polling.  ( interrupt is not used in the sense of suspending then returning to an activity, the function that is interrupted is over )
















[[Category:SmartTerminal]] [[Category:Arduino/RaspberryPi]]