.github/workflows/test-suite.yml

@@ -0,0 +1,40 @@
+# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
+# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
+
+name: Test Suite
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [3.7, 3.8, 3.9]
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install .
+        pip install flake8 pytest
+        if [ -f dev_requirements.txt ]; then pip install -r dev_requirements.txt; fi
+    - name: Lint with flake8
+      run: |
+        # stop the build if there are Python syntax errors or undefined names
+        flake8 openleadr/ --count --select=E9,F63,F7,F82 --show-source --statistics
+        # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
+        flake8 openleadr/ --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
+    - name: Test with pytest
+      run: |
+        pytest -vv test/

.gitignore

@@ -2,3 +2,11 @@ python_env/
 __pycache__
 pyopenadr.egg-info
 docs/_build/
+build/
+openleadr.egg-info/
+htmlcov/
+examples/
+dist/
+.pytest_cache/
+.ipynb_checkpoints/
+static/

CONTRIBUTING.md

@@ -0,0 +1,37 @@
+Developer Certificate of Origin
+Version 1.1
+
+Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
+1 Letterman Drive
+Suite D4700
+San Francisco, CA, 94129
+
+Everyone is permitted to copy and distribute verbatim copies of this
+license document, but changing it is not allowed.
+
+
+Developer's Certificate of Origin 1.1
+
+By making a contribution to this project, I certify that:
+
+(a) The contribution was created in whole or in part by me and I
+    have the right to submit it under the open source license
+    indicated in the file; or
+
+(b) The contribution is based upon previous work that, to the best
+    of my knowledge, is covered under an appropriate open source
+    license and I have the right under that license to submit that
+    work with modifications, whether created in whole or in part
+    by me, under the same open source license (unless I am
+    permitted to submit under a different license), as indicated
+    in the file; or
+
+(c) The contribution was provided directly to me by some other
+    person who certified (a), (b) or (c) and I have not modified
+    it.
+
+(d) I understand and agree that this project and the contribution
+    are public and that a record of the contribution (including all
+    personal information I submit with it, including my sign-off) is
+    maintained indefinitely and may be redistributed consistent with
+    this project or the open source license(s) involved.

MANIFEST.in

@@ -1,3 +1,5 @@
 include openleadr/templates/*.xml
 include openleadr/templates/parts/*.xml
-include openleadr/templates/signatures/*.xml
+include openleadr/schema/*.xsd
+include openleadr/schema/LICENSES.txt
+exclude test/*

README.md

@@ -1,3 +1,8 @@
+![Test Suite](https://github.com/OpenLEADR/openleadr-python/workflows/Test%20Suite/badge.svg)
+ [![Test Coverage](https://openleadr.org/coverage/badge.svg)](https://openleadr.org/coverage)
+ ![PyPI Downloads](https://img.shields.io/pypi/dm/openleadr?color=lightblue&label=PyPI%20Downloads)
+ [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/4317/badge)](https://bestpractices.coreinfrastructure.org/projects/4317)
+
 ![OpenLEADR](logo.png)
 
 OpenLEADR is a Python 3 module that provides a convenient interface to OpenADR
@@ -5,17 +10,52 @@ systems. It contains an OpenADR Client that you can use to talk to other OpenADR
 systems, and it contains an OpenADR Server (VTN) with convenient integration
 possibilities.
 
+It currently implements the OpenADR 2.0b specification.
+
 ## Documentation
 
-You can find documentation here: https://openleadr.elaad.io/docs
+You can find documentation here: https://openleadr.org/docs.
+
 
 ## Contributing
 
-At this moment, we're finishing off a first usable version. After version 0.5.0,
-new bug reports and pull requests are most welcome.
+If you'd like to help make OpenLEADR better, you can do so in the following ways:
+
+
+### File bug report or feature request
+
+We keep track of all bugs and feature requests on [Github Issues](https://github.com/openleadr/openleadr-python/issues). Please search the already close issues to see if your question was asked before.
+
+You're also very welcome to leave comments on existing issues with your ideas on how to improve OpenLEADR.
+
+
+### File a pull request
+
+We'd love for you to contribute code to the project. We'll take a look at all pull requests, no matter their state. Please note though, that we will only accept pull requests if they meet at least the following criteria:
+
+- Your code style is flake8 compliant (with a maximum line length of 127 characters)
+- You provide tests for your new code and your code, and you amend any previous tests that fail if they are impacted by your code change
+- Your pull request refers to an Issue on our issue tracker, so that we and other people can see what problem is being solved.
+- You sign off your commits (`git commit -s`), to indicate that your contribution complies with our license and does not violate anybody else's copyright.
+
+That said, please don't let the above requirements discourage you from filing a pull requests. If you don't meet all of the above requirements, we'll help you fix the remaining things to get it into shape.
+
+
+### Security issues
+
+Let it be clear that this code base is still in a development stage, and we don't yet recommend using it for mission critical applications or applications where security is paramount.
+
+That said, we do try to make OpenLEADR as secure as can be to work with. If you find a security vulnerability in OpenLEADR, please let us know at security AT openleadr DOT org. We will get back to you within 72 hours to follow up. We are committed to the following steps:
+
+- Security vulnerabilities with a known fix will be addressed as soon as possible. This means that work on other things will be put on hold until the security issue is fixed.
+- If a fix is not readily available, we will publish a warning that describes the vulnerable situation and, if possible, any mitigating steps that users of OpenLEADR can take.
+- After any security issue is fixed, we will publish information on it in the Changelog.
+
 
 ## Developing
 
+We recommend the following development setup for working with OpenLEADR (this is on Linux / macOS):
+
 ```bash
 git clone https://github.com/openleadr/openleadr-python
 cd openleadr-python
@@ -24,8 +64,9 @@ python3 -m venv python_env
 ./python_env/bin/pip3 install -r dev_requirements.txt
 ```
 
-## Running conformance tests
+To run the test suite, you can use the following command:
 
 ```bash
-./python_env/bin/python3 -m pytest test/conformance
+./python_env/bin/python3 -m pytest -v test/
 ```
+

VERSION

@@ -1 +1 @@
-0.4.2
+0.5.18

certificates/dummy_ca.crt

@@ -0,0 +1,32 @@
+-----BEGIN CERTIFICATE-----
+MIIFlzCCA3+gAwIBAgIUcDsCwW2aIuhvm5Zu+NrPI0nKNLkwDQYJKoZIhvcNAQEL
+BQAwWzELMAkGA1UEBhMCTkwxDjAMBgNVBAgMBU90aGVyMRswGQYDVQQKDBJPcGVu
+TEVBRFIgRHVtbXkgQ0ExHzAdBgNVBAMMFmR1bW15LWNhLm9wZW5sZWFkci5vcmcw
+HhcNMjAxMjIyMTIzMzQ0WhcNMzAxMjIwMTIzMzQ0WjBbMQswCQYDVQQGEwJOTDEO
+MAwGA1UECAwFT3RoZXIxGzAZBgNVBAoMEk9wZW5MRUFEUiBEdW1teSBDQTEfMB0G
+A1UEAwwWZHVtbXktY2Eub3BlbmxlYWRyLm9yZzCCAiIwDQYJKoZIhvcNAQEBBQAD
+ggIPADCCAgoCggIBALEn6YM4ohWaQtXDHoFgIlDkWC/zKxwzYlTa7PiLz0GEyw8d
+nRfSyki0PXSyyHEP70rS2MC4Vfvofd+Na4FYukJjUKc3bQLHIA+84YI3m6ksICF2
+NNn6jpCrOLMe5EcloufEtybVK7lU8xc0Jyaxmdom7px9tGcGrktaN2uyVrhsNz5u
+k4pi+z9SecAKx5UkCb2zGXd1Y1Ze8fQmDb0Bq3nBfep9qbTOrITHE8OV63T0/fXA
+fjqR+xBPpBLnmOE6DVx9AiIkLCFGL51thz7biL0BS2dyof8FTm1/eQ7KUHYCSWCL
+brAW6RyXL2tTYd0MxaJ7ugqgHKRQP/am4kCWzB2L7U9KmjrtfnDdk33wHGTFN+bz
+oKnClKY1gDju5PgIKNKt7m3ojMB7t0t9IZw/e9y4JgpcTGlowUyTD0XL2babDdTk
+wjwZmvWjAySRW+Ub4NT6WTVHco2NK9PAsUqzXsXDEok9RWP3dhfzfeOrncujzrMb
+7J8J6gpAKP+AmncNZIDaSTw8wr63gySwydgi4KgcWQuy85zro/fBNHWwUwWARYzI
+ETrgpl3SmSGBM8zeh5EoGvIyzoS2iD7cmbqsdKLavUPvvqBxKxgDb5DDs+x8s8Bk
+Hx54AKV3vf/1MhYMM1L19aTEEHvzX515l7SNvGBsDmpMlvCbQVqHYci5K6M9AgMB
+AAGjUzBRMB0GA1UdDgQWBBR9e/biv+uxPGMWwz4n8B+GvpTp0TAfBgNVHSMEGDAW
+gBR9e/biv+uxPGMWwz4n8B+GvpTp0TAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3
+DQEBCwUAA4ICAQACmAdt8q8rae+UGfqwi/sY8miO+Z6yLm8/oyDB5P6ftGH9x3C4
+a/6md1L02nf+Yz4R9SWkzKEtR87Y5j2Wqikz+aMXAHupCaq4Ei/nQ4DhyOrUpoxV
+2pAVBGfDJqf3DCRuTb/pKUUAAapEksWbvelyNRkuweVeHKDJbAveEbDCVagWP0NW
+G20g9X91MXGqDg6O7v1vw+aGHIgnK4hnb8y0O5mx9FSrg+UmINYIRC2Yg2igQ5T8
+RDfqDsDFyi0/VrAE4nz+v6Jtt6Wm9ksC+opIxIgiB2OZTnD6XQps3nUq+Egy1fZr
+ZUNPmimZiUS/0Dr/12hIx4/xGDVm4Dcv3mwyygEOTyCU9yNODalPoALCTf1WO2QR
+1tNaaPY8EOyJfTl/WsZ4qZ/WThSmrlVWrZI64jeu3dxBRLYf6aJEPMNA1FC+ikeH
+4p0JFo7idfK4mBj2GsUCzSAkntZZvnFCUVDji7KFRybJlKpoeOXxGmW51VS2SQx0
+YdX7y4XOUUKvm7HZUw6wIBC2hkPWSatpXu7DbKNuqyJVumfZvUs/zxGDorcCq4FD
+Nl7t/qDS7On4c1EphYIonsBnSszeRJ9plgu9c+dg5ypbyrwiHIejGb+A9heBtl7s
+XJBn3g1r8g89T8yWfj7uSBZSpDWJNDIRYhXMJkYXSOnD0NbucppAwqEyHg==
+-----END CERTIFICATE-----

certificates/dummy_ca.key

@@ -0,0 +1,51 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIJKAIBAAKCAgEAsSfpgziiFZpC1cMegWAiUORYL/MrHDNiVNrs+IvPQYTLDx2d
+F9LKSLQ9dLLIcQ/vStLYwLhV++h9341rgVi6QmNQpzdtAscgD7zhgjebqSwgIXY0
+2fqOkKs4sx7kRyWi58S3JtUruVTzFzQnJrGZ2ibunH20ZwauS1o3a7JWuGw3Pm6T
+imL7P1J5wArHlSQJvbMZd3VjVl7x9CYNvQGrecF96n2ptM6shMcTw5XrdPT99cB+
+OpH7EE+kEueY4ToNXH0CIiQsIUYvnW2HPtuIvQFLZ3Kh/wVObX95DspQdgJJYItu
+sBbpHJcva1Nh3QzFonu6CqAcpFA/9qbiQJbMHYvtT0qaOu1+cN2TffAcZMU35vOg
+qcKUpjWAOO7k+Ago0q3ubeiMwHu3S30hnD973LgmClxMaWjBTJMPRcvZtpsN1OTC
+PBma9aMDJJFb5Rvg1PpZNUdyjY0r08CxSrNexcMSiT1FY/d2F/N946udy6POsxvs
+nwnqCkAo/4Cadw1kgNpJPDzCvreDJLDJ2CLgqBxZC7LznOuj98E0dbBTBYBFjMgR
+OuCmXdKZIYEzzN6HkSga8jLOhLaIPtyZuqx0otq9Q+++oHErGANvkMOz7HyzwGQf
+HngApXe9//UyFgwzUvX1pMQQe/NfnXmXtI28YGwOakyW8JtBWodhyLkroz0CAwEA
+AQKCAgAw/TOC2QdhZ+4xhKqL5lS2/94vLFRwkPkRNBUxyh+/I4SvomXNr7nGjykr
+e0EYmup2S4YrDQ6iAbHFPytKconXT/V+uEIJ8Zy5HWdTBiOijZQ5DKIi4TnJYb/l
+MJa54fuBkhF/yJU1w/sRIJfvCE/eVsPHPK/FicBrEAChZIi8qRhByHw+WY7W/Oy0
+IYb9hCC5d6yEL08QFGNeO6Oy76JGoi46yRND758ffJnHjU62YgVUjy2Y7sN5yDw8
+ChVpuw7005DidhRKA0mphB9RT44pjhEXA0ku532/o++LGoFTkOBPtV8fjMZb5Kb1
+S/KVGTIR85sH0yz4d0So5Y9NgyXgU8QKyduGy4Duo6MOYhkcbCtX7PtAf5bwMBE7
+jkgMyrLOM3BfqRDUjUBPyPkOTs51t+gA64jt3a64CMteKar8pr+SA09AaViow5eq
+8UoqDpTaCJO6BLM8earMaqKccnUDF19vADzUOx7K++zScMJ2xRrdZECbCWmnc9Yv
+tSAkoXt5D5qYFIGVjazXOLdaDoc0Fa3bIlkRiuuFe1XM7oqxpQEgrjcM158Z30zR
+gsDZ2/x12VdRjZawfoGCeef9XzwvToq3OGcpA6Nx/H7wsDCsx9CNwtEkkpJHZPPa
+1goG0QKd0/aN3HjWqTszXEU1Tx1Nsl906mhf7Zrt4ctksNCOgQKCAQEA48pMEwPd
+0PoXBDKsVWTZLlidCnXcmt2PLnkr1pa2xjH7gi/MmRTX9vIfSvmFjow1JKz+5Qnc
+oEblBYKQYBiIn2WfUkZrSR8oen5gR9i717QOF9rbfaJ7L59Ph5XTgIYHOnePC0xR
+UkZrzStTMsHi3vh0eWwPJfitGdaGfwqfx/9q4Hv3BSwee/roR6cVxfn7HcDPYFY1
+mfvQqtsHnQryfVphX9st4NJtMn/6DBN/E37ZrcIqfvAcugCRHoLODS/DemCvjXUT
+J5FDO3qA3/XFeMTVsMyjkzI80oGiI1qt9pH2t+iS+/gqZ4FYcUWMiKnAejxTjTyz
+J3LdnkB+Om1mIQKCAQEAxxhW/4D/Fx/1JzNyXeJ+gUb/jG3yu1KTG4SbDQn4YwKJ
+uviZvzlWO1UnxUOLhthIhOKJtrb34CFja8T40GmGcR+L6P/zjh69YFZXlyyF/bBg
+mDIevKTfdTECUSm3n/1LOFmR8CZL9DR845wOIEczWDOJsnM8yeY/tG6IYnO/ks1e
+QYI9OHgK/auQ2KLMOW2C0RxbHSVvgxTt0uGKmrlYeMInKh5N/w1BSqRkz7zHme25
+jqSfWMhpTCDUaa6S6Bt9oWViH37id+Ji1HWlS5b7dXwqZvImw+GiY1X+VNv1BBu7
+x8kK5uvnzOenpsZjdMjDoXUYZu+HoG6C1U7METHhnQKCAQABxAOq8hC6GfYrtijX
+0JxOW6l790XqfWUquw238BsiliiY7b3sQdkatO3BKwX6AOQ4kI65P8ZSB7qmvEha
+NlZ4xdTiUmFqg69Qo8IjTG7IUUD6tluVMbk6uUVoi6TEDkXoEh0tvT20IY+cW4Wd
+kxsrF0nv1wKXDMJqvNr2CSML7dLqQou7oofp9hvu0kC89B8aL/Brkr9/nhAUAvag
+JQGULysqDwzf/UGTbqKFjXEiuYz+Z87khP/0ASUe02y9dW1SeYVi11F6sQmQYHa3
+RbRuJzhw0mVCMtV93DthgsZubts2ubrJ8TaC7uG8nwlj1c0EJYuQQLQyzhUhsOZJ
+laeBAoIBAQC6DajfRHEd7yNt6snpqouFzA9r8CNxoo0OnjE3UiXogKqtKzyiUrae
+48kysxhkfyHl7L29HecucIU0ZPIP8U1N0akxqF62ZUucB5P7FgRxOq5KDCxlJb4d
+ChucNVwACviMREf5IBDCuXosSd15lJAK0L7RIJeiJaVKvDB/sKNKUNjQZyFG8Ad1
+XysRB7HJyOkC+Xi8GAvJd3l4JIUeai6fSvDGj2NcAcsOMepzp3rwAhPxlS3EDiU4
+m0VlLVrjxSz18oRr4mtIlSq+WOKLbSC4fbwyUACh9O2H3wi5zIN/v1sQLHQfsD0C
+Y0lstOCkdik7bO3M4/Lenedt5yEUwISRAoIBAH29dhwxGkdqIOMC40xjK4qI2EXk
+ak5+d2n1YOZesjKjgxCkbqULrtLwUOn8GhYlIMnSIkmx5rLGIdQjSS7Kib7pUznd
+E9LujEhKEW/2R8MuULGoBU6cQ/glZkKl1Qyf+PzHXJXx1A3id0a1tOGbYdgrJHEO
+4iUAP3n5xUq6q3BJL3XWCfpGiCY+9Ls3O8fYIbK/Bo2B1Tyeeg3rXvHykpfwdvgw
+zhVex5UqRAwY2S4+QcbtfHd4WxLZC9KN0LB+Xrwe0uy+FKckWa0xxSlDXEJ94pGg
+jfz/EvYvgokNBTaxG3WJj5KPKD7zEH+fCTjEwjxdzwAQwwq9RJuo4A+ryx4=
+-----END RSA PRIVATE KEY-----

certificates/dummy_ca.srl

@@ -0,0 +1 @@
+5032FCA390507718BC6EB5EEF9F072D1F4D5E940

certificates/dummy_ven.crt

@@ -0,0 +1,25 @@
+-----BEGIN CERTIFICATE-----
+MIIEPzCCAicCFFAy/KOQUHcYvG617vnwctH01elAMA0GCSqGSIb3DQEBCwUAMFsx
+CzAJBgNVBAYTAk5MMQ4wDAYDVQQIDAVPdGhlcjEbMBkGA1UECgwST3BlbkxFQURS
+IER1bW15IENBMR8wHQYDVQQDDBZkdW1teS1jYS5vcGVubGVhZHIub3JnMB4XDTIw
+MTIyMjEyMzM0NFoXDTMwMTIyMDEyMzM0NFowXTELMAkGA1UEBhMCTkwxDjAMBgNV
+BAgMBU90aGVyMRwwGgYDVQQKDBNPcGVuTEVBRFIgRHVtbXkgVkVOMSAwHgYDVQQD
+DBdkdW1teS12ZW4ub3BlbmxlYWRyLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEP
+ADCCAQoCggEBALzQdvgRTggTIA8oBpY2WU+fxxnXJUDV315CeRiE+pg15GNFCe6a
+T0wyBtCCoQgUTIAlhnzQUak23bRwAxLZo0voKcGK4EHJqt70hZpO/7Lj4OGeMTxr
+SzAP4pG/pgPSfGyJxMAXkuPOUtjmg7I+Bt+LHkqUIQpxLQqlkV7p9prdbrkS3rUU
+9s1XPdjoAMMUL8el1knXRT34ioJdTdQlhxL0e4nbDViV4oOFpkyS7S0zkQ9MN/pe
+JxdoDk/MTW0TWcIhWSIqSkgJwEBK6NtoqqQ6vs6op6luoJUmW3ut/rXam66KXRyu
+2NFLgIZpN7wuQ+BJOWSLK7IOVg9dwjjHLpsCAwEAATANBgkqhkiG9w0BAQsFAAOC
+AgEAIyw6xubLbKuFkv4gDSyJh0NFt30dEK0wXdHvar4kdFA3I7t8vlRm93h/PRq3
+K87tqrUly+C59lLFgckPcbXFbouaaaabvegfpmK9U5beYk6v7d5KmTLEK8+IplAw
+xkCM8OAxpj5FZ5P0hPl5+4MnDfAkTcsrpUsNekn31OO97NuGoIj6rqYb0rLVPccI
+cb9zjoBbSR7XGsuc3afoYebJOccyGY/FhWEmV7vQUDpO8dyUqUsS1p5Okzv4PhDk
+gnZM9YDR4ShJ+84yos9iYJLAm/rNOezM2YvaJCxzuO18nIXf1qK/ahEkMXrgRw/B
+dXhAzxt6/W0e2Zbm20jf26NGPAjpA6KhuC7pUYs1cLalC1B4uctkWZixVO5GbPmO
+jXTUcqCPTPFvWlqZDWKgldny3xlOEWs4Kn6rCI3g2qc8tmN04U1CHQaZ2O7Ekhw7
+Orns5K0im+gZt9yKrbzj08etex1wVB6G8MKXjDWY5Dv1HbMZwKj/g+ZkGEvRHBtK
+93vdlVvEAkqdL1eiA8NV65/B5rTbxTG85IlUMKc5CCWlLwI3ifl4mnAZQ4B1Z5Eh
+Vbr+OZp5ywrp1HoRtkUBP+NWm9JD/sMdbOJMTBQT2oeUoOmWxWBRYNjV6rHb0Djl
+y5V7Qbt4EPgFgXPCEUTbEgkL4K1QOvGOWf2LeMHJos6HLwM=
+-----END CERTIFICATE-----

certificates/dummy_ven.csr

@@ -0,0 +1,17 @@
+-----BEGIN CERTIFICATE REQUEST-----
+MIICojCCAYoCAQAwXTELMAkGA1UEBhMCTkwxDjAMBgNVBAgMBU90aGVyMRwwGgYD
+VQQKDBNPcGVuTEVBRFIgRHVtbXkgVkVOMSAwHgYDVQQDDBdkdW1teS12ZW4ub3Bl
+bmxlYWRyLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALzQdvgR
+TggTIA8oBpY2WU+fxxnXJUDV315CeRiE+pg15GNFCe6aT0wyBtCCoQgUTIAlhnzQ
+Uak23bRwAxLZo0voKcGK4EHJqt70hZpO/7Lj4OGeMTxrSzAP4pG/pgPSfGyJxMAX
+kuPOUtjmg7I+Bt+LHkqUIQpxLQqlkV7p9prdbrkS3rUU9s1XPdjoAMMUL8el1knX
+RT34ioJdTdQlhxL0e4nbDViV4oOFpkyS7S0zkQ9MN/peJxdoDk/MTW0TWcIhWSIq
+SkgJwEBK6NtoqqQ6vs6op6luoJUmW3ut/rXam66KXRyu2NFLgIZpN7wuQ+BJOWSL
+K7IOVg9dwjjHLpsCAwEAAaAAMA0GCSqGSIb3DQEBCwUAA4IBAQCDo861OwtXYjTe
+xwDwhW3VAKxLA2cZ63UbTKt7qySlbiTgXGiNLy1GZMZD38mispijXw7qCymvnXrS
+TiG7ni/dBAolg+ZSE4lungibKMNJ8XJKXC0911X+0eCVJdwuvYXS4QfPLYS3KmL8
+C9K/QYwhdXj702h4g8kTJ5Ycc67wLTZ7ag44KZMAvVBpvrMl2gHK0HXFeckYjCI+
+twG730nacxamBbA0Hhcd1rEnHEjcxQPrYnuILsbEDpiSOGdXPfYLOpbgJwwmp4Zu
+b+x/yl+0lrE5wwFG/2mOq/SUDns9nl4o1Ro7T7C2A0U+fAf+8Uo8nrUShIj0ikbc
+wigEGWrO
+-----END CERTIFICATE REQUEST-----

certificates/dummy_ven.key

@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEpQIBAAKCAQEAvNB2+BFOCBMgDygGljZZT5/HGdclQNXfXkJ5GIT6mDXkY0UJ
+7ppPTDIG0IKhCBRMgCWGfNBRqTbdtHADEtmjS+gpwYrgQcmq3vSFmk7/suPg4Z4x
+PGtLMA/ikb+mA9J8bInEwBeS485S2OaDsj4G34seSpQhCnEtCqWRXun2mt1uuRLe
+tRT2zVc92OgAwxQvx6XWSddFPfiKgl1N1CWHEvR7idsNWJXig4WmTJLtLTORD0w3
++l4nF2gOT8xNbRNZwiFZIipKSAnAQEro22iqpDq+zqinqW6glSZbe63+tdqbropd
+HK7Y0UuAhmk3vC5D4Ek5ZIsrsg5WD13COMcumwIDAQABAoIBAF+Jt/jzgKFTVBB3
+N0YAgBZrCWqI01/IGDrwtadzef1Un5ifUVQ7Hk62rX4J6wNUihT5Z+B15CwUCACK
+APQjzZ3V/nLhG3IOYfhoj8WxnW5eIebnjZA91hCeqQ0IhS8/7RdaaoSsKPY96uu3
+UAH7oqywDwa7hzBqbdkKR9FX3yEidXbZB5Dq84fO0gqig2X8NgVl9IeA6LUh6sSN
+fxHcvbYLT2lDHcpl7Oamk8JpK/bIVptj2lZvxPs49V/a08OV6YwCrkBjraiPoVFt
+B9sLigc6rNq9uUc4jdxwyMOWaS9uKlhZV0dvI6NJaHIHF10X/Slk3ZaNfiJnV0bq
+fYLNgDkCgYEA5XfSv1iYUl2JVxwT0BPG+kFRccWXXe5pmVEzmCtVT/eLvh6N3wE3
+Iwpy3PZjqGcIPlFJCZfDQJpBnjSHY/QMgOX24RdH6MCMlg+RW23Xcj+8AuHTOgCX
+pPsfGm+jwkc9tgbSsduEUKngf5Yd30ugy1uuAcX3uSsIAVpIJw0wTh8CgYEA0qVN
+8B2miOrHoE4q2I7hiB2IM5eLx+vZ03l3MIba20nAaVRWoTmcrrdej9yse5P+xMJ8
+hl3lJbEdA9gK6k2mq+Jo2AIZfjjp1sS+ZAKU6Or4chDF7TdP4jvs45Y/izcK0J9E
+PmPdKY8A31ScRTFHCkBkmVms1TY9ajiEh76lWAUCgYEApRebYZ1tIb04JZsGyiqg
+esZpwVAmwibYhLz+QNnUbE8ulB9JdQtbzvMihsUiGDPgo205/hPZH26cDSW/zvLz
+1/0brQBh9RwrSX9z1fLmEcW3D9/HZ7CracBetVdi21EEHiU0i0/jF2HRKhon7dJs
+okKYo5/5xZgnD0oUJTyA54MCgYEAvf6r1bBozYY2mLjonHwDoKpCd4ZxZdmtl7kv
+cG2yaaiUDG0t1i4IzO5INKpuSOisGvzxJKD8VoryCM2MytlPRCnrNyptpBPhlv3O
+XJaXiZ6miPvoCpahTwWOHZkfp4n2D0YYX83jZeC+gLHoYeCYmv6JvmfMJGPP8UcZ
+AvdKW6ECgYEAsZtSYvwdR8EQ48fNqsRRP9oiDFzHdd6nK3lrnuicu5jghO7yX1tG
+ATPEKjCChbJehTtmaS2DqKjwJE6yX7crdGH13n6hr8coi1aDSdmwUbDdNmq+HFOt
+OA9k1AvUIfmzXu8VlnClfD+8uCt7mlNt9kg3EbJ6Lpbgk6rDddvgqvI=
+-----END RSA PRIVATE KEY-----

certificates/dummy_vtn.crt

@@ -0,0 +1,25 @@
+-----BEGIN CERTIFICATE-----
+MIIEPzCCAicCFFAy/KOQUHcYvG617vnwctH01ek/MA0GCSqGSIb3DQEBCwUAMFsx
+CzAJBgNVBAYTAk5MMQ4wDAYDVQQIDAVPdGhlcjEbMBkGA1UECgwST3BlbkxFQURS
+IER1bW15IENBMR8wHQYDVQQDDBZkdW1teS1jYS5vcGVubGVhZHIub3JnMB4XDTIw
+MTIyMjEyMzM0NFoXDTMwMTIyMDEyMzM0NFowXTELMAkGA1UEBhMCTkwxDjAMBgNV
+BAgMBU90aGVyMRwwGgYDVQQKDBNPcGVuTEVBRFIgRHVtbXkgVlROMSAwHgYDVQQD
+DBdkdW1teS12dG4ub3BlbmxlYWRyLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEP
+ADCCAQoCggEBAKFyy5udqOzgypjsL9ZcZg9AuQHRk2KNIQD5NSew5PHlM/s4sxwR
+Tc0c/BhXXlwfWQYnxjjm3XgyBblB4wOMF8LRImsHHM4DbkiMvG42zg2ooCTdCFFN
+L+8ORRG5VpCajdPrarm78pt7Jyqmz752mT8fdr3A6aEHmgGCCAMQDejQshSN7+Rk
+dezjWr5QRSxfraKrcKmAx59Bxwh92D+KZUBYBSta4eNj0/Wea8EAQxwBVqVpU5hE
+qrBl2QiGuZ4pAG0QL5pgbnR5WH6PcwnwNgJrHW44+BvPcau/aYZWICaW6DYceZHk
+lnu6LbLH1KQZZiH7VIPTh3/oC5Yo3D4Ox3UCAwEAATANBgkqhkiG9w0BAQsFAAOC
+AgEAh8jFnbprXCADlGMXXsReyJ7FNG1cbndKF8dZL+TIBlzoBZn8P/hT85rzo2PI
+PQwDy2b3SPwKN1RCwfKLDGdDwTI65IQXSgihxNdxPMWVNjwc0VQ5vkHUrW7OPUBl
+1HuYxOTViZ9LOQKVJf/gCodZqurhUeR/sJgUO8kHhj/HfceWxqP3/EnKAptFpy8Q
+Ruyuqa/mSYbEURFIbRP7vzQianlAaCoyy8i5JucLvLgPZRmCYsIHoz/8Oeq2MrVq
+TCV2dCPlA2VXTpkWaRoV1gsrEU127f8sWFqpodbfNVheclzEm2IS8MILg+FMH/w7
+jZe3W/jzCh87r8uwPsg1ZizqW8KhWTM/l/e+B9CHX/uH66wYAE/nKjz1HDRkr7HF
+1TX4HdgZRt675iyck2+GhRX2RV2XHmr2xCszddv7mbPEFNklCGKN6c9k1UOKjmgo
+528VpRXlTjsmX8bVRN1xu0Q6vFFRKmylt5aD1mxirMcQZSaMk65CVfPQTaKxL2Q5
+r1ACvITCoAGfuRAwrdt7hGAX3s1ukBr1S+0S21wXzy2zKOsJh6KO7UTYdQ4pFIMk
+9kUWRrJgIT/59z29pg2FPuRkIyLaN/kiw3ROpPA7OtQf9gpMnBT7uSAxxrqBY/me
+F2sGabuTrafs8naLmafstcx2VPQRIt7XM/YXnwe7vAilrSc=
+-----END CERTIFICATE-----

certificates/dummy_vtn.csr

@@ -0,0 +1,17 @@
+-----BEGIN CERTIFICATE REQUEST-----
+MIICojCCAYoCAQAwXTELMAkGA1UEBhMCTkwxDjAMBgNVBAgMBU90aGVyMRwwGgYD
+VQQKDBNPcGVuTEVBRFIgRHVtbXkgVlROMSAwHgYDVQQDDBdkdW1teS12dG4ub3Bl
+bmxlYWRyLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKFyy5ud
+qOzgypjsL9ZcZg9AuQHRk2KNIQD5NSew5PHlM/s4sxwRTc0c/BhXXlwfWQYnxjjm
+3XgyBblB4wOMF8LRImsHHM4DbkiMvG42zg2ooCTdCFFNL+8ORRG5VpCajdPrarm7
+8pt7Jyqmz752mT8fdr3A6aEHmgGCCAMQDejQshSN7+RkdezjWr5QRSxfraKrcKmA
+x59Bxwh92D+KZUBYBSta4eNj0/Wea8EAQxwBVqVpU5hEqrBl2QiGuZ4pAG0QL5pg
+bnR5WH6PcwnwNgJrHW44+BvPcau/aYZWICaW6DYceZHklnu6LbLH1KQZZiH7VIPT
+h3/oC5Yo3D4Ox3UCAwEAAaAAMA0GCSqGSIb3DQEBCwUAA4IBAQBp91POIKdL4i2I
+40eqqJsnRpmT7Y15N98RJ2Wkd0b0dvTjTNiPrQCrLdnoJLmfyGGaTv9V0i29eTIu
+cqEKk0Hn7CEqEGX3mtn/GtP4mDZ/OoYqyApET+XYCwDV0VwHc85/mZWIkjVgg3KS
+1JP08BClNEkMDVMhACC1N+R3HWDRBTRs6cm76Zu5HYwxIt3OlElg82zGFJ+DtYNg
+sjq4EzdwFQqUyab6ZxwMiyDz8xTO9kDLSn1PRgR/lASmUEtA1ADia+X+ziNbftDB
+AHKFuPjjAkFk50zax9k11ZIQew7nu3+QHc3lVuWOZg7Y748AEkEpzmpLC649Ibxc
+7b+0gbLA
+-----END CERTIFICATE REQUEST-----

certificates/dummy_vtn.key

@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEowIBAAKCAQEAoXLLm52o7ODKmOwv1lxmD0C5AdGTYo0hAPk1J7Dk8eUz+ziz
+HBFNzRz8GFdeXB9ZBifGOObdeDIFuUHjA4wXwtEiawcczgNuSIy8bjbODaigJN0I
+UU0v7w5FEblWkJqN0+tqubvym3snKqbPvnaZPx92vcDpoQeaAYIIAxAN6NCyFI3v
+5GR17ONavlBFLF+toqtwqYDHn0HHCH3YP4plQFgFK1rh42PT9Z5rwQBDHAFWpWlT
+mESqsGXZCIa5nikAbRAvmmBudHlYfo9zCfA2Amsdbjj4G89xq79phlYgJpboNhx5
+keSWe7otssfUpBlmIftUg9OHf+gLlijcPg7HdQIDAQABAoIBAEyJnGbopjWuF73J
+cqA/645dk5d+IE3/M98/RWjMnqziiPMXHSo4NdcAX662dKBdqBmX74v4PpQFutrp
+llKPLpCIDrYIiCpOTBEOzyG8z5CAGXOAaboJSqkc7beKUrR44LXPjCgDJu94wceh
+jyjvFpVrOCKa+ucBMIx4dY3mJYHx9SS79APYIJazpgaVBIJS879iqrKqgOlifouo
+AmYfUrJw52hUyb20Xs2FW05jbd2Wvb1AmfXch2nVbKP12wwDQ5sD1C1QgyU/KFNg
+YclfvkCMRv7G5/vois87qheUSokaaipZHgolQHskdPvy9aPaBkDdjx2zo4T9Od2d
+JsD5zuECgYEA0yecr/gKMX+w8f77/29Ux30IgIcoy1yKtxsuicsoSr5qQ+m5yfiM
+SXzrMhLY+Lsh42IpctbgwsLsN4XwenOYFD/5k2RcxGsQz7HNMf1UCWCD2LXl8VPg
+drr3MOvZvSBcqzWDJpWE8326SKecf5ypX9KBaNElR1p1aJmFavtzMg0CgYEAw7ys
+XAOe05OiHgtMkUkhbQMVek2ohATsC1HaxYaagYXDdBKrYl4QuojhsMeEfOu1C8Fd
+Kx15L9llcI4C5Uvzbn5Jh7WZy+M027yLobHrViIu7MKw0jzgJquuDI9kXfhaKj4Q
+n7p8gOa01oe2kgVjf9jofV2sZy5Q2uDUSNOp2QkCgYEAvXOwGQ5yjuDjsOy3ywJn
+zaj4ZOFgD75TU2CXC9j0qMNZ8t8U7AsOS61CFSZl+B1mlW5wg/IZrYVYjaSmFCf8
+zkZsNft5ZF4vsjn0QqLpsJQhF+J0kmdQYRb1TLaAITmVC6QyrP7AT/uBlUiVmMXG
+DgyAQbxNN00JRLVhyJAdVk0CgYAdZEp5cq251ZRlcIrp0hJr3SevetPQJzEOrhbs
+zD6NLnngByGnHrriV8WUFxGk9Hv2LO4BmGZnMfzSfvCeX95I/DusXi+e9xor2M8a
+062j/HQRZ1bH6w45oFP9XNbUaYOYjkNOyOEDNiF3iV9348lCHF8k3BkUUVcg1tvp
+p6b8iQKBgG6kno9yHuS3pAaJ7sbjNMywHN4torDv6Fk3vbNgFkxEEKYTjVGGr66h
+m6oxteOvBL2WpFthOtkN01FrDu1G3J3ZreXFfPhkOU7OwCgb18DAOUzZoHkinXYB
+7yqkUx1PHw902acZ186r7INr6Tm1pBnKD7cKsrKT2LcGKAIW3lyr
+-----END RSA PRIVATE KEY-----

certificates/generate_certificates.sh

@@ -0,0 +1,27 @@
+set -e
+
+echo "Please make sure the line RANDFILE = ... is commented out in your /etc/ssl/openssl.conf."
+
+echo "Generating the CA key"
+openssl genrsa -out dummy_ca.key 4096
+
+echo "Generating the CA cert"
+openssl req -x509 -new -subj "/C=NL/ST=Other/O=OpenLEADR Dummy CA/CN=dummy-ca.openleadr.org" -nodes -key dummy_ca.key -sha256 -days 3650 -out dummy_ca.crt
+
+echo "Generating the VTN key"
+openssl genrsa -out dummy_vtn.key 2048
+
+echo "Generating the VTN Certificate Signing Request"
+openssl req -new -sha256 -key dummy_vtn.key -subj "/C=NL/ST=Other/O=OpenLEADR Dummy VTN/CN=dummy-vtn.openleadr.org" -out dummy_vtn.csr
+
+echo "Signing the VTN CSR, generating the VTN certificate"
+openssl x509 -req -in dummy_vtn.csr -CA dummy_ca.crt -CAkey dummy_ca.key -CAcreateserial -out dummy_vtn.crt -days 3650 -sha256
+
+echo "Generating the VEN key"
+openssl genrsa -out dummy_ven.key 2048
+
+echo "Generating the VEN Certificate Signing Request"
+openssl req -new -sha256 -key dummy_ven.key -subj "/C=NL/ST=Other/O=OpenLEADR Dummy VEN/CN=dummy-ven.openleadr.org" -out dummy_ven.csr
+
+echo "Signing the VTN CSR, generating the VEN certificate"
+openssl x509 -req -in dummy_ven.csr -CA dummy_ca.crt -CAkey dummy_ca.key -CAcreateserial -out dummy_ven.crt -days 3650 -sha256

docs/Makefile

@@ -4,7 +4,7 @@
 # You can set these variables from the command line, and also
 # from the environment for the first two.
 SPHINXOPTS    ?=
-SPHINXBUILD   ?= sphinx-build
+SPHINXBUILD   ?= ../python_env/bin/sphinx-build
 SOURCEDIR     = .
 BUILDDIR      = _build
 

docs/_static/css/custom.css

@@ -1,69 +0,0 @@
-@font-face {
-    font-family: 'charter-webfont';
-    src: url('../fonts/charter_regular.woff') format('woff');
-    font-weight: normal;
-    font-style: normal;
-}
-
-@font-face {
-    font-family: 'charter-webfont';
-    src: url('../fonts/charter_italic.woff') format('woff');
-    font-weight: normal;
-    font-style: italic;
-}
-
-@font-face {
-    font-family: 'charter-webfont';
-    src: url('../fonts/charter_bold.woff') format('woff');
-    font-weight: bold;
-    font-style: normal;
-}
-
-@font-face {
-    font-family: 'charter-webfont';
-    src: url('../fonts/charter_bold_italic.woff') format('woff');
-    font-weight: bold;
-    font-style: italic;
-}
-
-html{
-    border-top: 5px solid #785b9c;
-    border-bottom: 5px solid #785b9c;
-    min-height: 100vh;
-    box-sizing: border-box;
-}
-
-body {
-    font-family: 'Charter', 'charter-webfont', 'Georgia', sans-serif;
-}
-
-h1, h2, h3 {
-    font-family: 'Charter', 'charter-webfont', 'Georgia', sans-serif;
-}
-
-div.body pre {
-    font-size: 80%;
-}
-
-div.sphinxsidebarwrapper p.logo{
-    margin-bottom: 20px;
-    text-align: left;
-}
-
-div.sphinxsidebarwrapper h1.logo{
-    display: none;
-}
-
-div.body a{
-    color: #785b9c;
-    text-decoration: none;
-    border-bottom: none;
-}
-
-div.body a:hover{
-    color: white;
-    background-color: #785b9c;
-    padding: 0px 2px;
-    margin: 0px -2px;
-    border-radius: 3px;
-}

docs/api/openleadr.rst

@@ -28,6 +28,14 @@ openleadr.errors module
    :undoc-members:
    :show-inheritance:
 
+openleadr.fingerprint module
+----------------------------
+
+.. automodule:: openleadr.fingerprint
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 openleadr.messaging module
 --------------------------
 

docs/client.rst

@@ -4,19 +4,60 @@ Client
 
 An OpenADR Client (Virtual End Node or VEN) usually represents an entity that owns controllable devices. This can be electric vehicles, generators, wind turbines, refrigerated warehouses, et cetera. The client connects to a server, usualy representing a utility company, to discuss possible cooperation on energy usage throughout the day.
 
-In your application, you mostly only have to deal with two things: Events and Reports.
+
+.. _client_example:
+
+Example VEN
+===========
+
+A straightforward example of an OpenADR VEN, which has one report and an event handler, would look like this:
+
+.. code-block:: python3
+
+    import asyncio
+    from datetime import timedelta
+    from openleadr import OpenADRClient
+
+    async def collect_report_value():
+        # This callback is called when you need to collect a value for your Report
+        return 1.23
+
+    async def handle_event(event):
+        # This callback receives an Event dict.
+        # You should include code here that sends control signals to your resources.
+        return 'optIn'
+
+    # Create the client object
+    client = OpenADRClient(ven_name='myven', vtn_url='http://some-vtn.com/OpenADR2/Simple/2.0b')
+
+    # Add the report capability to the client
+    client.add_report(callback=collect_report_value,
+                      resource_id='device001',
+                      measurement='voltage',
+                      sampling_rate=timedelta(seconds=10))
+
+    # Add event handling capability to the client
+    client.add_handler('on_event', handle_event)
+
+    # Run the client in the Python AsyncIO Event Loop
+    loop = asyncio.get_event_loop()
+    loop.create_task(client.run())
+    loop.run_forever()
+
+In the sections below, we'll go into more detail.
+
 
 .. _client_events:
 
 Dealing with Events
 ===================
 
-Events are informational or instructional messages from the server (VTN) which inform you of price changes, request load reduction, et cetera. Whenever there is an Event for your VEN, your ``on_event`` handler will be called with the event as its ``payload`` parameter.
+Events are informational or instructional messages from the server (VTN) which inform you of price changes, request load reduction, et cetera. Whenever there is an Event for your VEN, your ``on_event`` handler will be called with the event is dict-form as its first parameter.
 
 The Event consists of three main sections:
 
 1. A time period for when this event is supposed to be active (``active_period``)
-2. A list of Targets to which the Event applies (``target``). This can be the VEN as a whole, or specific groups, assets, geographic areas, et cetera that this VEN represents.
+2. A list of Targets to which the Event applies (``targets``). This can be the VEN as a whole, or specific groups, assets, geographic areas, et cetera that this VEN represents.
 3. A list of Signals (``signals``), which form the content of the Event. This can be price signals, load reduction signals, et cetera. Each signal has a name, a type, multiple Intervals that contain the relative start times, and some payload value for the client to interpret.
 
 After you evaluate all these properties, you have only one decision to make: Opt In or Opt Out. Your handler must return either the string ``optIn`` or ``optOut``, and OpenLEADR will see to it that your response is correctly formatted for the server.
@@ -25,47 +66,74 @@ Example implementation:
 
 .. code-block:: python3
 
-    from openadr import OpenADRClient
-
-    async def on_event(payload):
+    async def on_event(event):
         # Check if we can opt in to this event
-        start_time = payload['events'][0]['active_period']['dtstart']
-        duration = payload['events'][0]['active_period']['duration']
-
-        await can_we_do_this(from_time=payload[''])
+        first_signal = event['event_signals'][0]
+        intervals = first_signal['intervals']
+        target = event['target']
+        ...
         return 'optIn'
 
+An example event dict might look like this:
 
-.. _client_reports:
+.. code-block:: python3
 
-Dealing with Reports
-====================
+    {
+        'event_id': '123786-129831',
+        'active_period': {'dtstart': datetime.datetime(2020, 1, 1, 12, 0, 0, tzinfo=timezone.utc),
+                          'duration': datetime.timedelta(minutes=30)}
+        'event_signals': [{'signal_name': 'simple',
+                           'signal_type': 'level',
+                           'intervals': [{'dtstart': datetime.datetime(2020, 1, 1, 12, 0, 0, tzinfo=timezone.utc),
+                                          'duration': datetime.timedelta(minutes=10),
+                                          'signal_payload': 1},
+                                          {'dtstart': datetime.datetime(2020, 1, 1, 12, 10, 0, tzinfo=timezone.utc),
+                                          'duration': datetime.timedelta(minutes=10),
+                                          'signal_payload': 0},
+                                          {'dtstart': datetime.datetime(2020, 1, 1, 12, 20, 0, tzinfo=timezone.utc),
+                                          'duration': datetime.timedelta(minutes=10),
+                                          'signal_payload': 1}],
+       'targets': [{'resource_id': 'Device001'}],
+       'targets_by_type': {'resource_id': ['Device001']}
+    }
+
+Please note that you can access the targets in two ways, which may be useful if there are more than one target:
+
+1. As a list of Target dicts
+2. As a dictionary of targets, grouped by target type.
+
+For example:
 
-The VTN Server will most like want to receive some reports like metering values or availability status from you.
-Providing reports
------------------
+.. code-block:: python3
 
-If you tell OpenLEADR what reports you are able to provide, and give it a handler that will retrieve those reports from your own systems, OpenLEADR will make sure that the server receives the reports it asks for and at the requested interval.
+    {
+        'event_id': 'event123',
+        # ...
+        # As a list of Target dicts
+        'targets': [{'resource_id': 'resource01'},
+                    {'resource_id': 'resource02'},
+                    {'group_id': 'group01'},
+                    {'group_id': 'group02'}],
+        # Grouped by target type
+        'targets_by_type': {'resource_id': ['resource01', 'resource02'],
+                            'group_id': ['group01', 'group02']}
+    }
 
-For example: you can provide 15-minute meter readings for an energy meter at your site. You have a coroutine set up like this:
+It is up to you which you want to use.
 
-.. code-block:: python3
 
-    async def get_metervalue():
-        current_value = await meter.read()
-        return current_value
+.. _client_reports:
 
-And you configure this report in OpenLEADR using an :ref:`oadrReportDescription` dict:
+Dealing with Reports
+====================
 
-.. code-block:: python3
+The VTN Server will most likely want to receive some reports like metering values or availability status from you.
 
-    async def main():
-        client = OpenADRClient(ven_name='MyVEN', vtn_url='https://localhost:8080/')
-        report_description = {''}
-        client.add_report({'report'})
+You can easily add reporting capabilities to your OpenADRClient object using the ``client.add_report`` method. In this method, you supply a callback function that will retrieve the current value for that measurement, as well as the resource_id, the measurement (like 'voltage', 'power', 'temperature', et cetera), optionally a unit and scale, and a sampling rate at which you can support this metervalue.
 
-The only thing you need to provide is the current value for the item you are reporting. OpenLEADR will format the complete :ref:`oadrReport` message for you.
+OpenLEADR will then offer this report to the VTN, and if they request this report from you, your callback function will automatically be called when needed.
 
+Please see the :ref:`reporting` section for detailed information.
 
 
 .. _client_signing_messages:
@@ -104,3 +172,15 @@ You can validate incoming messages against a public key.
 This will automatically validate check that incoming messages are signed by the private key that belongs to the provided (public) certificate. If validation fails, you will see a Warning emitted, but the message will not be delivered to your handlers, protecting you from malicious messages being processed by your system. The sending party will see an error message returned.
 
 You should use both of the previous examples combined to secure both the incoming and the outgoing messages.
+
+
+.. _client_polling_jitter:
+
+A word on polling
+=================
+
+The OpenADR polling mechanism is very robust; there is very little chance that the client misses an important message. The downside is that there is some wasted bandwith (from polling when no relevant message is available from the VTN), and there is the risk of unnecessary VTN overload if all VENs poll synchronously.
+
+To mitigate the last point, the OpenLEADR VEN will, by default, 'jitter' the pollings by up to +/- 10% or +/- 5 minutes (whichever is smallest). The same goes for delivering the reports (the data collection will still happen on synchronized moments).
+
+If you don't want to jitter the polling requests on your VEN, you can disable this by passing ``allow_jitter=False`` to your ``OpenADRClient`` constructor.

docs/conf.py

@@ -54,7 +54,15 @@ autoclass_content = 'both'
 # a list of builtin themes.
 #
 html_theme = 'alabaster'
-html_logo = 'logo-tall.png'
+# html_logo = 'logo-tall.png'
+html_theme_options = {
+    'logo': 'logo-tall.png',
+    'logo_name': False,
+    'github_user': 'openleadr',
+    'github_repo': 'openleadr-python',
+    'font_family': 'sans-serif',
+    'font_size': 8
+}
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
@@ -62,5 +70,4 @@ html_logo = 'logo-tall.png'
 html_static_path = ['_static']
 
 
-def setup(app):
-    app.add_css_file('css/custom.css')  # may also be an URL
+

docs/examples.rst

@@ -1,50 +0,0 @@
-.. _examples:
-
-=====================
-Ready-to-Run Examples
-=====================
-
-This page contains examples for OpenLEADR:
-
-.. _client_example:
-
-Client Example
-==============
-
-This example sets up a minimal OpenADR Client (Virtual End Node):
-
-.. code-block:: python3
-
-    from openleadr import OpenADRClient
-    import asyncio
-
-    async def main():
-        client = OpenADRClient(ven_name="Device001", vtn_url="http://localhost:8080/OpenADR2/Simple/2.0b")
-        client.on_event = handle_event
-        await client.run()
-
-    async def handle_event(event):
-        """
-        This coroutine will be called
-        whenever there is an event to be handled.
-        """
-        print("There is an event!")
-        print(event)
-        return 'optIn'
-
-    loop = asyncio.get_event_loop()
-    loop.create_task(main())
-    loop.run_forever()
-
-
-
-
-.. _server_example:
-
-Server Example
-==============
-
-.. _server_with_gui_example:
-
-Server with GUI Example
-=======================

docs/index.rst

@@ -30,6 +30,11 @@ License
 
 This project is licensed under the Apache 2.0 license.
 
+Development and contributing
+============================
+
+The source code of this project can be found on `GitHub <https://github.com/openleadr>`_. Feature requests, bug reports and pull requests are most welcome and can be posted there.
+
 Library Installation
 ====================
 
@@ -37,7 +42,7 @@ Library Installation
 
    $ pip install openleadr
 
-OpenLEADR is compatible with Python 3.6+
+OpenLEADR is compatible with Python 3.7 and higher.
 
 Getting Started
 ===============
@@ -50,7 +55,7 @@ Client example::
     async def main():
         client = OpenADRClient(ven_name="Device001",
                                vtn_url="http://localhost:8080/OpenADR2/Simple/2.0b")
-        client.on_event = handle_event
+        client.add_handler('on_event', handle_event)
         await client.run()
 
     async def handle_event(event):
@@ -69,7 +74,6 @@ Client example::
 
 This will connect to an OpenADR server (indicated by the vtn_url parameter), handle registration, start polling for events and reports, and will call your coroutines whenever an event or report is created for you.
 
-We have more examples available over at the :ref:`examples` page.
 
 
 Table of Contents
@@ -80,14 +84,14 @@ Table of Contents
    :maxdepth: 2
 
    features
-   openadr
    client
    server
-   examples
-   representations
+   reporting
+   logging
    message_signing
    roadmap
    API Reference <api/modules>
+   representations
 
 Representations of OpenADR payloads
 ===================================

docs/logging.rst

@@ -0,0 +1,66 @@
+=======
+Logging
+=======
+
+OpenLEADR uses the standard Python Logging facility. Following the `Python guidelines <https://docs.python.org/3/howto/logging.html#configuring-logging-for-a-library>`_, no default handlers are configured, but you can easily get going if you want to:
+
+Log to standard output
+----------------------
+
+To print logs to your standard output, you can use the following convenience method:
+
+.. code-block:: python3
+
+    import openleadr
+    import logging
+    openleadr.enable_default_logging(level=logging.INFO)
+
+Which is the same as:
+
+.. code-block:: python3
+
+    import logging
+    logger = logging.getLogger('openleadr')
+    handler = logging.StreamHandler()
+    handler.setLevel(logging.INFO)
+    logger.addHandler(handler)
+
+
+Setting the logging level
+-------------------------
+
+You can set different logging levels for the logging generated by OpenLEADR:
+
+.. code-block:: python3
+
+    import logging
+    logger = logging.getLogger('openleadr')
+    logger.setLevel(logging.WARNING)
+
+The different `logging levels <https://docs.python.org/3/library/logging.html#levels>`_ are:
+
+.. code-block:: python3
+
+    logging.DEBUG
+    logging.INFO
+    logging.WARNING
+    logging.ERROR
+    logging.CRITICAL
+
+Redirecting logs to a different file
+------------------------------------
+
+To write logs to a file:
+
+.. code-block:: python3
+
+    import logging
+    logger = logging.getLogger('openleadr')
+
+    handler = logging.FileHandler('mylogfile.txt')
+    logger.addHandler(logger)
+
+More info
+---------
+
+Detailed info on logging configuration can be found on `the official Python documentation <https://docs.python.org/3/library/logging.html>`_.

docs/message_signing.rst

@@ -42,3 +42,18 @@ To prevent an attacker from simple re-playing a previous message (for instance,
 
 OpenLEADR automatically generates and validates these portions of the signature. Signed messages that do not contain a ReplayProtect element are rejected, as required by the OpenADR specification.
 
+
+Certificate Fingerprints
+------------------------
+
+In order for the opposing party to verify that you ar who you say you are, they need the fingerprint of your public key that you are using. You can get this fingerprint in two ways:
+
+1. It will be printed to your stdout when you start your client or server. If you don't want this, your can turn it off by passing `show_fingerprint=False` to the OpenADRClient() or OpenADRServer() constructors.
+2. You can use an included commandline tool to show the fingerprint of any PEM certificate:
+
+.. code-block:: bash
+
+    pip3 install --user openleadr
+    fingerprint path/to/cert.pem
+
+The fingerprint is not sensitive material; you can distribute it via an unsecured method, or you can publish it as a DNS-record for example.

docs/openadr.rst

@@ -1,40 +0,0 @@
-==============
-OpenADR Basics
-==============
-
-If you are coming to this module and are not (yet) familiar with the OpenADR protocol, read this. Of course, you should also consult the documentation from `the OpenADR website <https://www.openadr.org>`_.
-
-High-level overview
-===================
-
-OpenADR is a protocol that allows a server (called a Virtual Top Node or VTNs) to communicate 'Events' to connected clients (Called Virtual End Nodes or VENs). These Events are usually energy-related instructions, for instance to temporarily increase or reduce the power consumption by one or more devices represented by the VEN, or te inform the VEN that prices are about to change. The VEN periodically (typically every 10 seconds or so) sends a Poll request to the VTN to check if there are new events for them.
-
-The VEN decides whether or not to comply with the request in the Event, and sends an Opt In or Opt Out response to the VTN.
-
-In order to track what happens after, there is a Reports mechanism in place that allows the VEN and the VTN to agree on what data should be reported, and to report this data at a requested interval.
-
-Although multiple transport formats are supported (HTTP and XMPP), OpenADR is designed for only the VTN to be public-accessible, with the VENs possibly being behind NAT or firewalls. All communications are therefore initiated by the client (VEN), and the server can request additional messages from the client in its response to the original request.
-
-
-.. _registration:
-
-Registration
-============
-
-(Information on the Registration procedures)
-
-
-
-.. _openadr_events:
-
-Events
-======
-
-(Information on the Events procedures)
-
-.. _openadr_reports:
-
-Reports
-=======
-
-(Information on the Reports procedures)

docs/reporting.rst

@@ -0,0 +1,332 @@
+.. _reporting:
+
+=========
+Reporting
+=========
+
+Your VEN can provide periodic reports to the VTN. These reports usually contain metering data or status information.
+
+A brief overview of reports in OpenADR
+--------------------------------------
+
+Reports can describe many measurable things. A report description contains:
+
+- A ReportName, of which the following are predefined in OpenADR:
+   - ``TELEMETRY_USAGE``: for providing meter reading or other instantanious values
+   - ``TELEMETRY_STATUS``: for providing real-time status updates of availability
+   - ``HISTORY_USAGE``: for providing a historic metering series
+   - ``HISTORY_GREENBUTTON``: for providing historic data according to the GreenButton format
+- A ReportType, which indicates what the data represents. There are many options predefined, like ``READING``, ``USAGE``, ``DEMAND``, ``STORED_ENERGY``, et cetera. You can find all of them in the ``enums.REPORT_TYPE`` enumeration.
+- A ReadingType, which indicates the way in which the data is collected or possibly downsampled. For instance, ``DIRECT_READ``, ``NET``, ``ALLOCATED``, ``SUMMED``, ``MEAN``, ``PEAK``, et cetera.
+- A Sampling Rate, which defines the rate at which data can or should be collected. When configuring / offering reports from the VEN, you can set a minimum and maximum sampling rate if you wish, to let the VTN choose its preferred sampling rate.
+- A so-called ItemBase, which indicates the quantity you are reporting, like Voltage, Current or Energy. In OpenLEADR, this is called ``measurement``.
+- A unit of measure, usually linked to the ``measurement``, like ``A`` for ampere, or ``V`` for volt.
+- A Resource ID, which indicates which of the resources this report applies to. It's probably one of your controllable devices.
+
+You configure which reports you can deliver, and the VEN offers these to the VTN. The VTN makes a selection and requests reports from the VEN to be sent at regular intervals. The VEN then starts collecting data and sending reports.
+
+Offering reports (VEN to VTN)
+-----------------------------
+
+
+Basic telemetry reports
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Say you have two devices, 'Device001' and 'Device002', which can each offer measurments of Voltage and Current at a samplerate of 10 seconds. You would register these reports as follows:
+
+.. code-block:: python3
+
+    import openleadr
+    from functools import partial
+
+    async def main():
+        client = openleadr.OpenADRClient()
+        # Add reports
+        client.add_report(callback=partial(read_current, device='Device001'),
+                          report_specifier_id='AmpereReport',
+                          resource_id='Device001',
+                          measurement='Current',
+                          sampling_rate=timedelta(seconds=10),
+                          unit='A')
+        client.add_report(callback=partial(read_current, device='Device002'),
+                          report_specifier_id='AmpereReport',
+                          resource_id='Device002',
+                          measurement='Current',
+                          sampling_rate=timedelta(seconds=10),
+                          unit='A')
+        client.add_report(callback=partial(read_voltage, device='Device001'),
+                          report_specifier_id='VoltageReport',
+                          resource_id='Device001',
+                          measurement='Voltage',
+                          sampling_rate=timedelta(seconds=10),
+                          unit='V')
+        client.add_report(callback=partial(read_voltage, device='Device002'),
+                          report_specifier_id='VoltageReport',
+                          resource_id='Device002',
+                          measurement='Voltage',
+                          sampling_rate=timedelta(seconds=10),
+                          unit='V')
+        await client.run()
+
+    async def read_voltage(device):
+        """
+        Retrieve the voltage value from the given 'device'.
+        """
+        v = await interface.read(device, 'voltage') # Dummy code
+        return v
+
+    async def read_current(device):
+        """
+        Retrieve the current value from the given 'device'.
+        """
+        a = await interface.read(device, 'current') # Dummy code
+        return a
+
+
+The VTN can request TELEMETRY_USAGE reports to be delivered at the sampling rate, or it can request them at a lower rate. For instance, it can request 15-minute readings, delivered every 24 hours. By default, openLEADR handles this case by incrementally building up the report during the day, calling your callback every 15 minutes and sending the report once it has 24 hours worth of values.
+
+If, instead, you already have a system where historic data can be extracted, and prefer to use that method instead, you can configure that as well.
+
+The two requirements for this kind of data collection are:
+
+1. Your callback must accept arguments named ``date_from``, ``date_to`` and ``sampling_interval``
+2. You must specify ``data_collection_mode='full'`` when adding the report.
+
+Here's an example:
+
+.. code-block:: python3
+
+    import openleadr
+
+    async def main():
+        client = openleadr.OpenADRClient(ven_name='myven', vtn_url='http://some-vtn.com')
+        client.add_report(callback=load_data,
+                          data_collection_mode='full',
+                          report_specifier_id='AmpereReport',
+                          resource_id='Device001',
+                          measurement='current',
+                          sampling_rate=timedelta(seconds=10),
+                          unit='A')
+
+    async def load_data(date_from, date_to, sampling_rate):
+        """
+        Function that loads data between date_from and date_to, sampled at sampling_rate.
+        """
+        # Load data from a backend system
+        result = await database.get("""SELECT time_bucket('15 minutes', datetime) as dt, AVG(value)
+                                         FROM metervalues
+                                        WHERE datetime BETWEEN %s AND %s
+                                        GROUP BY dt
+                                        ORDER BY dt""")
+        # Pack the data into a list of (datetime, value) tuples
+        data = result.fetchall()
+
+        # data should look like:
+        # [(datetime.datetime(2020, 1, 1, 12, 0, 0), 10.0),
+        #  (datetime.datetime(2020, 1, 1, 12, 15, 0), 9.0),
+        #  (datetime.datetime(2020, 1, 1, 12, 30, 0), 11.0),
+        #  (datetime.datetime(2020, 1, 1, 12, 45, 0), 12.0)]
+        return data
+
+
+Historic data reports
+~~~~~~~~~~~~~~~~~~~~~
+
+.. note::
+    Historic reports are not yet implemented into OpenLEADR. Please follow updates in `this issue on GitHub <https://github.com/OpenLEADR/openleadr-python/issues/18>`_.
+
+You can also configure historic reports, where the VTN can at any time request data from a specified time interval and granularity. For historic reports, you must have your own data collection system and the provided callback must have the signature:
+
+.. code-block:: python3
+
+    async def get_historic_data(date_from, date_to, sampling_interval)
+
+
+An example for configuring historic reports:
+
+.. code-block:: python3
+
+    import openleadr
+    from functools import partial
+
+    async def main():
+        client = openleadr.OpenADRClient(ven_name='myven', vtn_url='http://some-vtn.com')
+        client.add_report(callback=partial(get_historic_data, device_id='Device001'),
+                          report_name='HISTORY_USAGE',
+                          report_specifier_id='AmpereHistory',
+                          resource_id='Device001',
+                          measurement='current'
+                          sampling_rate=timedelta(seconds=10),
+                          unit='A')
+
+Note that you have to override the default ``report_name`` compared to the previous examples.
+
+
+Requesting Reports (VTN to VEN)
+-------------------------------
+
+The VTN will receive an ``oadrRegisterReport`` message. Your handler ``on_register_report`` will be called for each report that is offered. You inspect the report description and decide which elements from the report you wish to receive.
+
+Using the compact format
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The compact format provides an abstraction over the actual encapsulation of reports. If your ``on_register_report`` handler has the following signature, if will be called using the simple format:
+
+.. code-block:: python3
+
+    async def on_register_report(ven_id, resource_id, measurement, unit, scale,
+                                 min_sampling_interval, max_sampling_interval):
+        if want_report:
+            return (callback, sampling_interval, report_interval)
+        else:
+            return None
+
+The ``callback`` refers to a function or coroutine that will be called when data is received.
+The ``sampling_interval`` is a ``timedelta`` object that contains the interval at which data is sampled by the client.
+The ``report_interval`` is optional, and contains a ``timedelta`` object that indicates how often you want to receive a report. If you don't specify a ``report_interval``, you will receive each report immediately after it is sampled.
+
+This mechanism allows you to specify, for instance, that you want to receive 15-minute sampled values every 24 hours.
+
+For more information on the design of your callback function, see the :ref:`receiving_reports` section below.
+
+Using the full format
+~~~~~~~~~~~~~~~~~~~~~
+
+If you want full control over the reporting specification, you implement an ``on_register_report`` handler with the following signature:
+
+.. code-block:: python3
+
+    async def on_register_report(report):
+        # For each report description (identified by their r_id)
+        # you want to received, return a callback and sampling rate
+
+        return [(callback_1, r_id_1, sampling_rate_1),
+                (callback_2, r_id_2, sampling_rate_2)]
+
+The Report object that you have to inspect looks like this:
+
+.. code-block:: python3
+
+    {'report_specifier_id': 'AmpereHistory',
+     'report_name': 'METADATA_TELEMETRY_USAGE',
+     'report_descriptions': [
+
+            {'r_id': 'abc346-de6255-2345',
+             'report_type': 'READING',
+             'reading_type': 'DIRECT_READ',
+             'report_subject': {'resource_ids': ['Device001']},
+             'report_data_source': {'resource_ids': ['Device001']},
+             'sampling_rate': {'min_period': timedelta(seconds=10),
+                               'max_period': timedelta(minutes=15),
+                               'on_change': False},
+             'measurement': {'item_name': 'current',
+                             'item_description': 'Current',
+                             'item_units': 'A',
+                             'si_scale_code': None},
+
+            {'r_id': 'd2e352-126ae2-1723',
+             'report_type': 'READING',
+             'reading_type': 'DIRECT_READ',
+             'report_subject': {'resource_ids': ['Device002']},
+             'report_data_source': {'resource_ids': ['Device002']},
+             'sampling_rate': {'min_period': timedelta(seconds=10),
+                               'max_period': timedelta(minutes=15),
+                               'on_change': False},
+             'measurement': {'item_name': 'current',
+                             'item_description': 'Current',
+                             'item_units': 'A',
+                             'si_scale_code': None}
+
+        ]
+    }
+
+.. note:: The ``report_name`` property of a report gets prefixed with ``METADATA_`` during the ``register_report`` step. This indicates that it is a report without any data. Once you get the actual reports, the ``METADATA_`` prefix will not be there.
+
+Your handler should read this, and make the following choices:
+
+- Which of these reports, specified by their ``r_id``, do I want?
+- At which sampling rate? In other words: how often should the data be sampled from the device?
+- At which reporting interval? In other words: how ofter should the collected data be packed up and sent to me?
+- Which callback should be called when I receive a new report?
+
+Your ``on_register_report`` handler thus looks something like this:
+
+.. code-block:: python3
+
+    import openleadr
+
+    async def store_data(data):
+        """
+        Function that stores data from the report.
+        """
+
+    async def on_register_report(resource_id, measurement, unit, scale, min_sampling_period, max_sampling_period):
+        """
+        This is called for every measurement that the VEN is offering as a telemetry report.
+        """
+        if measurement == 'Voltage':
+            return store_data, min_sampling_period
+
+    async def main():
+        server = openleadr.OpenADRServer(vtn_id='myvtn')
+        server.add_handler('on_register_report', on_register_report)
+
+Your ``store_data`` handler will be called with the contents of each report as it comes in.
+
+You have two options for receiving the data:
+
+1. Receive the entire oadrReport dict that contains the values as we receive it.
+2. Receive only the r_id and an iterable of ``(datetime.datetime, value)`` tuples for you to deal with.
+
+
+Delivering Reports (VEN to VTN)
+-------------------------------
+
+Report values will be automatically collected by running your provided callbacks. They are automatically packed up and sent to the VTN at the requested interval.
+
+Your callbacks should return either a single value, representing the most up-to-date reading,
+or a list of ``(datetime.datetime, value)`` tuples. This last type is useful when providing historic reports.
+
+This was already described in the previous section on this page.
+
+
+.. receiving_reports::
+
+Receiving Reports (VTN)
+-----------------------
+
+When the VEN delivers a report that you asked for, your handlers will be called to deal with it.
+
+Instead of giving you the full Report object, your handler will receive the iterable of ``(datetime.datetime, value)`` tuples.
+
+If your callback needs to know other metadata properties at runtime, you should add those as default arguments during the request report phase. For instance:
+
+.. code-block:: python3
+
+    from functools import partial
+
+    async def receive_data(data, resource_id, measurement):
+        for timestamp, value in data:
+            await database.execute("""INSERT INTO metervalues (resource_id, measurement, timestamp, value)
+                                         VALUES (%s, %s, %s, %s)""", (resource_id, measurement, dt, value))
+
+
+    async def on_register_report(resource_id, measurement, unit, scale, min_sampling_rate, max_sampling_rate):
+        prepared_callback = partial(receive_data, resource_id=resource_id, measurement=measurement)
+        return (callback, min_sampling_rate)
+
+The ``partial`` function creates a version of your callback with default parameters filled in.
+
+
+Identifying a data stream
+-------------------------
+
+Reports in OpenADR carry with them the following identifiers:
+
+- ``reportSpecifierID``: the id that the VEN assigns to this report
+- ``rID``: the id that the VEN assigns to a specific data stream in the report
+- ``reportRequestID``: the id that the VTN assigns to its request of a report
+- ``reportID``: the id that the VEN assigns to a single copy of a report
+
+The ``rID`` is the most specific identifier of a data stream. The ``rID`` is part of the ``oadrReportDescription``, along with information like the measurement type, unit, and the ``resourceID``.
+

docs/roadmap.rst

@@ -10,10 +10,277 @@ Upcoming releases
 -----------------
 
 ======= ================================== ====================
-Version Main features                      Target date
+Version Main features                      Target timeframe
 ======= ================================== ====================
-0.4.0   Implement XML message signing      September 15th, 2020
-0.5.0   Implement reporting                October 1st, 2020
-0.6.0   Implement XMPP transport           November 1st, 2020
+0.6.0   Implement XMPP transport           January 2020
 1.0.0   Certification by OpenADR Alliance  T.B.A.
 ======= ================================== ====================
+
+.. _changelog:
+
+Changelog
+---------
+
+openleadr 0.5.18
+~~~~~~~~~~~~~~~~
+
+Released: 22 January 2021
+
+Bug fixes:
+
+- OpenLEADR now correctly communicates all active and upcoming events in the correct order on every eiRequestEvent or, if a new event was added, on the next oadrPoll action from the client.
+
+Improvements:
+
+- Some more value checking in the reporting mechanism
+- Various restructurings of the code surrounding report registration and delivery
+
+openleadr 0.5.17
+~~~~~~~~~~~~~~~~
+
+Released: 5 January 2021
+
+Bug fixes:
+
+- reportRequestID is now correctly set to 0 in the oadrRegisterReport message
+- The Content-Type header is now correctly set on all VEN requests, and the VTN will check for it.
+- x-LoadrControlPercentOffset contained a typo
+- The oadrRegisterReport reportDescription would contain an invalid default MarketContext, which is now fixed
+
+openleadr 0.5.16
+~~~~~~~~~~~~~~~~
+
+Released: 15 December 2020
+
+Bug fixes:
+
+- Various bug fixes surrounding report registration. If your handlers returned only None or some incompatible values, it should now be much more graceful and helpful about it.
+- Some bug fixes surrounding the placement of the resourceID within the oadrRegisterReport messages.
+- Fixed parsing datetimes that don't contain microseconds; it is now also compatible with datetimes that only provide milliseconds.
+
+
+openleadr 0.5.15
+~~~~~~~~~~~~~~~~
+
+Released: 15 December 2020
+
+Bug fixes:
+
+- Restore Python 3.7 compatibility (got broken in 0.5.14)
+
+New features:
+
+- You can now use a future instead of a callback function or coroutine when adding an event. This allows you to add and event and await the response in a single place.
+- You can now add events that don't require a response, and the VEN will no longer respond to events that don't expect a response. In this case, your on_event handler may still, but does not need to, return an opt status. The returned opt status will be ignored in that case.
+
+
+openleadr 0.5.14
+~~~~~~~~~~~~~~~~
+
+Released: 15 December 2020
+
+New features:
+
+- Added support for a status callback to the server.add_raw_event method, just like the ``server.add_event`` method.
+
+API changes:
+
+- Removed the stale server.run() method and replaced it with a coroutine that does the same as ``server.run_async()``.
+
+Bug fixes:
+
+- Removed a naming inconsistency in the objects.ActivePeriod object.
+- Silently cancel running tasks when stopping the client or server.
+- Implemented the full duration regex for parsing timedeltas.
+- Various improvements to the test suite and some stale code cleanup.
+
+Other changes:
+
+- Changed the way openleadr is packaged, dropped the setup-time inclusion of the VERSION file.
+- OpenLEADR is now also available under the previous name pyopenadr. A new version of pyopenadr will be released in lockstep with new versions of openleadr. pyopenadr only contains an ``__init__`` file that does ``from openleadr import *``.
+
+openleadr 0.5.13
+~~~~~~~~~~~~~~~~
+
+Released: 10 December 2020
+
+New features:
+
+- This version adds support for the oadrRequestEvent on the VTN side.
+
+Bug fixes:
+
+- Fixed a bug where messages from the VTN that did not contain an EiResponse field caused a KeyError in the VEN (#33).
+
+
+openleadr 0.5.12
+~~~~~~~~~~~~~~~~
+
+Released: 10 December 2020
+
+New features:
+
+- Events now cycle through the correct 'far', 'near', 'active', 'completed'.
+- The Client now implements the ``on_update_event handler``, so that you can catch these event updates separately from the regular event messages.
+- Added support for the ramp_up_period parameter on the ``server.add_event`` method.
+
+Bug fixes:
+
+- The OpenADRServer would block ``oadrPoll`` requests when no internal messages were available. This has been corrected.
+- Some left-over ``print()`` statements have been removed.
+- Nonce caching was badly broken in a previous version, this has now been fixed.
+
+
+
+openleadr 0.5.11
+~~~~~~~~~~~~~~~~
+
+Released: 3 December 2020
+
+New features:
+
+- This update makes the list of Targets available as a dictionary of targets grouped by their type.
+- You can now add Targets to events in multiple ways (``target``, ``targets``, and ``targets_by_type``).
+
+Changes:
+
+- The member names of the 'measurement' objects or dicts have been changed to be consistent everywhere:
+    - item_name -> name
+    - item_description -> description
+    - item_units -> unit
+    - si_scale_code -> scale
+    This way, the parameters to client.add_report() are consistent with the Measurement object and the dicts that are passed around.
+    Additionally, there is extra validation to prevent sending invalid measurements, and hints to correct any mistakes.
+
+
+openleadr 0.5.10
+~~~~~~~~~~~~~~~~
+
+Released: 1 December 2020
+
+Bug fixes:
+
+- The on_created_event handler is now expected to receive the parameters (ven_id, event_id, opt_type). This was already in the docs, but not yet in the actual implementation. This has now been fixed.
+
+openleadr 0.5.9
+~~~~~~~~~~~~~~~
+
+Released: 1 December 2020
+
+New features:
+
+- Added the ven fingerprint to the registration_info dict for the ``on_create_party_registration`` handler. This allows the VTN to verify the fingerprint upon registration, even when the VEN does not have a venID yet.
+
+Changes:
+- Converted the OpenADRServer.add_raw_event method to a normal (synchronous) method.
+
+Bug fixes:
+- The EiResponse.response_code would not always show up correctly, this is now fixed.
+
+openleadr 0.5.8
+~~~~~~~~~~~~~~~
+
+Released: 30 November 2020
+
+New features:
+
+- Added the ``ven_id`` to the list of parameters for the ``on_register_report`` handler, so that this handler can know which VEN is registering reports
+- Updated documentation to reflect the current API of OpenLEADR
+
+openleadr 0.5.7
+~~~~~~~~~~~~~~~
+
+Released: 27 November 2020
+
+Bugs fixed:
+
+- Fixed a typo in the EventService.on_created_event placeholder function
+
+openleadr 0.5.5
+~~~~~~~~~~~~~~~
+
+Released: 23 November 2020
+
+New features:
+
+- Message signing now uses the correct C14n algorithm, as required by OpenADR
+- Preliminary VEN support for multiple events in one DistributeEvent message
+
+openleadr 0.5.4
+~~~~~~~~~~~~~~~
+
+Released: 23 November 2020
+
+New features:
+
+- Preliminary support for TELEMETRY_STATUS reports
+- Changed the server.add_event to be a normal function (not a coroutine), which allows you to call it from a synchronous function if needed.
+
+openleadr 0.5.3
+~~~~~~~~~~~~~~~
+
+Released: 20 November 2020
+
+New features:
+
+- Support for custom units in Reports is back, and is now compliant with the XML Schema.
+- Support for specifying the measurement (unit) in an EventSignal is added, and builds on the work of the report units.
+
+
+openleadr 0.5.2
+~~~~~~~~~~~~~~~
+
+Released: 19 November 2020
+
+
+Bug fixes:
+
+- The 'full' report data collection mode now works correctly
+- Various codestyle improvements and cleanup
+
+Known issues:
+
+- The support for out-of-schema measurements in repors has been removed, because they would not pass XML validation. We are exploring solutions to this problem. Track the progress here: `Issue #20 <https://github.com/OpenLEADR/openleadr-python/issues/20>`_
+
+openleadr 0.5.1
+~~~~~~~~~~~~~~~
+
+Released: 19 November 2020
+
+New features:
+
+- When using SSL connections, the client will provide server side SSL certificates. The VTN will verify the fingerprint of these certificates against the known fingerprint for that ven. This should complete the VEN authentication process.
+
+
+Bug fixes:
+
+- Report messages now validate according to the XML schema. A few corrections were made to the templates from version 0.5.0.
+
+
+Known issues:
+
+- The support for out-of-schema measurements in repors has been removed, because they would not pass XML validation. We are exploring solutions to this problem. Track the progress here: `Issue #20 <https://github.com/OpenLEADR/openleadr-python/issues/20>`_
+
+
+openleadr 0.5.0
+~~~~~~~~~~~~~~~
+
+Released: 16 November 2020
+
+First release to pypi.org.
+
+New features:
+
+- This release implements reporting functionality into the client and the server. This is a major new area of functionality for OpenLEADR.
+
+openleadr 0.4.0
+~~~~~~~~~~~~~~~
+
+Released: 16 November 2020
+
+Only released to git.
+
+New features:
+
+- This release implements XML Message Signing for client and servers.
+

docs/server.rst

@@ -6,6 +6,82 @@ Server
 
 If you are implementing an OpenADR Server ("Virtual Top Node") using OpenLEADR, read this page.
 
+.. _server_example:
+
+1-minute VTN example
+====================
+
+Here's an example of a server that accepts registrations from a VEN named
+'ven_123', requests all reports that it offers, and creates an Event for this
+VEN.
+
+.. code-block:: python3
+
+    import asyncio
+    from datetime import datetime, timezone, timedelta
+    from openleadr import OpenADRServer, enable_default_logging
+    from functools import partial
+
+    enable_default_logging()
+
+    async def on_create_party_registration(registration_info):
+        """
+        Inspect the registration info and return a ven_id and registration_id.
+        """
+        if registration_info['ven_name'] == 'ven123':
+            ven_id = 'ven_id_123'
+            registration_id = 'reg_id_123'
+            return ven_id, registration_id
+        else:
+            return False
+
+    async def on_register_report(ven_id, resource_id, measurement, unit, scale,
+                                 min_sampling_interval, max_sampling_interval):
+        """
+        Inspect a report offering from the VEN and return a callback and sampling interval for receiving the reports.
+        """
+        callback = partial(on_update_report, ven_id=ven_id, resource_id=resource_id, measurement=measurement)
+        sampling_interval = min_sampling_interval
+        return callback, sampling_interval
+
+    async def on_update_report(data, ven_id, resource_id, measurement):
+        """
+        Callback that receives report data from the VEN and handles it.
+        """
+        for time, value in data:
+            print(f"Ven {ven_id} reported {measurement} = {value} at time {time} for resource {resource_id}")
+
+    async def event_response_callback(ven_id, event_id, opt_type):
+        """
+        Callback that receives the response from a VEN to an Event.
+        """
+        print(f"VEN {ven_id} responded to Event {event_id} with: {opt_type}")
+
+    # Create the server object
+    server = OpenADRServer(vtn_id='myvtn')
+
+    # Add the handler for client (VEN) registrations
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+
+    # Add the handler for report registrations from the VEN
+    server.add_handler('on_register_report', on_register_report)
+
+    # Add a prepared event for a VEN that will be picked up when it polls for new messages.
+    server.add_event(ven_id='ven_id_123',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[{'dtstart': datetime(2021, 1, 1, 12, 0, 0, tzinfo=timezone.utc),
+                                 'duration': timedelta(minutes=10),
+                                 'signal_payload': 1}],
+                     callback=event_response_callback)
+
+    # Run the server on the asyncio event loop
+    loop = asyncio.get_event_loop()
+    loop.create_task(server.run())
+    loop.run_forever()
+
+Read on for more details!
+
 .. _server_registration:
 
 Registration
@@ -15,16 +91,16 @@ If a client (VEN) wants to register for the first time, it will go through a Reg
 
 .. admonition:: Implementation Checklist
 
-    1. Create a handler that decides what to do with new registrations, based on their ``venID``.
+    1. Create a handler that decides what to do with new registrations, based on their registration info.
 
 
 The client will send a :ref:`oadrQueryRegistration` message. The server will respond with a :ref:`oadrCreatedPartyRegistration` message containing a list of its capabilities, notably the implemented OpenADR protocol versions and the available Transport Mechanisms (HTTP and/or XMPP).
 
 The client will then usually send a :ref:`oadrCreatePartyRegistration` message, in which it registers to a specific OpenADR version and Transport Method. The server must then decide what it wants to do with this registration.
 
-In the case that the registration is accepted, the VTN will generate a RegistrationID for this VEN and respond with a :ref:`oadrCreatedPartyRegistration` message.
+In the case that the registration is accepted, the VTN will generate a venID and a RegistrationID for this VEN and respond with a :ref:`oadrCreatedPartyRegistration` message.
 
-In your application, when a VEN sends a :ref:`oadrCreatePartyRegistration` request, it will call your ``on_register_party`` handler. This handler must somehow look up what to do with this request, and respond with a ``registration_id``.
+In your application, when a VEN sends a :ref:`oadrCreatePartyRegistration` request, it will call your ``on_create_party_registration`` handler. This handler must somehow look up what to do with this request, and respond with a ``ven_id, registration_id`` tuple.
 
 Example implementation:
 
@@ -33,25 +109,27 @@ Example implementation:
     from openleadr.utils import generate_id
 
     async def on_create_party_registration(payload):
-        ven_id = payload['ven_id']
+        ven_name = payload['ven_name']
         # Check whether or not this VEN is allowed to register
         result = await database.query("""SELECT COUNT(*)
                                            FROM vens
-                                          WHERE ven_id = ?""",
-                                      (payload['ven_id'],))
+                                          WHERE ven_name = ?""",
+                                      (payload['ven_name'],))
         if result == 1:
             # Generate an ID for this registration
+            ven_id = generate_id()
             registration_id = generate_id()
 
             # Store the registration in a database (pseudo-code)
             await database.query("""UPDATE vens
-                                       SET registration_id = ?
-                                     WHERE ven_id = ?""",
-                                 (registration_id, ven_id))
+                                       SET ven_id = ?
+                                       registration_id = ?
+                                     WHERE ven_name = ?""",
+                                 (ven_id, registration_id, ven_name))
 
             # Return the registration ID.
             # This will be put into the correct form by the OpenADRServer.
-            return registration_id
+            return ven_id, registration_id
 
 .. _server_events:
 
@@ -60,89 +138,143 @@ Events
 
 The server (VTN) is expected to know when it needs to inform the clients (VENs) of certain events that they must respond to. This could be a predicted shortage or overage of available power in a certain electricity grid area, for example.
 
-The VTN must determine when VENs are relevant and which Events to send to them. The next time the VEN polls for new messages (using a :ref:`oadrPoll` or :ref:`oadrRequestEvent` message), it will send the Event in a :ref:`oadrDistributeEvent` message to the client. The client will then evaluate whether or not it indends to comply with the request, and respond with an :ref:`oadrCreatedEvent` message containing an optStatus of ``'optIn'`` or ``'optOut'``.
+The easiest way to supply events to a VEN is by using OpenLEADR's built-in message queing system. You simply add an event for a ven using the ``server.add_event`` method. You supply the ven_id for which the event is required, as well as the ``signal_name``, ``signal_type``, ``intervals`` and ``targets``. This will build an event object with a single signal for a VEN. If you need more flexibility, you can alternatively construct the event dictionary yourself and supply it directly to the ``add_raw_event`` method.
 
-.. admonition:: Implementation Checklist
+The VEN can decide whether to opt in or opt out of the event. To be notified of their opt status, you supply a callback handler which will be called when the VEN has responded to the event request.
 
-    In your application, the creation of Events is completely up to you. OpenLEADR will only call your ``on_poll`` handler with a ``ven_id``. This handler must be able to either retrieve the next event for this VEN out of some storage or queue, or make up the Event in real time.
+.. code-block:: python3
 
-    - ``on_created_event(payload)`` handler is called whenever the VEN sends an :ref:`oadrCreatedEvent` message, probably informing you of what they intend to do with the event you gave them.
-    - ``on_request_event(ven_id)``: this should return the next event (if any) that you have for the VEN. If you return ``None``, a blank :ref:`oadrResponse` will be returned to the VEN.
-    - ``on_request_report(ven_id)``: this should return then next report (if any) that you have for the VEN. If you return None, a blank :ref:`oadrResponse` will be returned to the VEN.
-    - ``on_poll(ven_id)``: this should return the next message in line, which is usually either a new :ref:`oadrUpdatedReport` or a :ref:`oadrDistributeEvent` message.
+    from openleadr import OpenADRServer
+    from functools import partial
+    from datetime import datetime, timezzone
 
+    async def event_callback(ven_id, event_id, opt_status):
+        print(f"VEN {ven_id} responded {opt_status} to event {event_id}")
 
-The Event consists of three main sections:
+    server = OpenADRServer(vtn_id='myvtn')
+    event_id = server.add_event(ven_id='ven123',
+                                signal_name='simple',
+                                signal_type='level',
+                                intervals=[{'dtstart': datetime(2020. 1, 1, 12, 0, 0, tzinfo=timezone.utc),
+                                            'signal_payload': 1},
+                                            {'dtstart': datetime(2020. 1, 1, 12, 15, 0, tzinfo=timezone.utc),
+                                            'signal_payload': 0}],
+                                target=[{'resource_id': 'Device001'}],
+                                callback=event_callback)
 
-1. A time period for when this event is supposed to be active
-2. A list of Targets to which the Event applies. This can be the VEN as a whole, or specific groups, assets, geographic areas, et cetera that this VEN represents.
-3. A list of Signals, which form the content of the Event. This can be price signals, load reduction signals, et cetera. Each signal has a name, a type, multiple Intervals that contain the relative start times, and some payload value for the client to interpret.
 
+Alternatively, you can use the handy constructors in ``openleadr.objects`` to format parts of the event:
 
+.. code-block:: python3
 
-.. _server_reports:
+    from openleadr import OpenADRServer
+    from openleadr.objects import Target, Interval
+    from datetime import datetime, timezone
+    from functools import partial
+
+    async def event_callback(ven_id, event_id, opt_status):
+        print(f"VEN {ven_id} responded {opt_status} to event {event_id}")
+
+    server = OpenADRServer(vtn_id='myvtn')
+    event_id = server.add_event(ven_id='ven123',
+                                signal_name='simple',
+                                signal_type='level',
+                                intervals=[Interval(dtstart=datetime(2020, 1, 1, 12, 15, 0, tzinfo=timezone.utc),
+                                                    signal_payload=0),
+                                           Interval(dtstart=datetime(2020, 1, 1, 12, 15, 0, tzinfo=timezone.utc),
+                                                    signal_payload=1)]
+                                target=[Target(resource_id='Device001')],
+                                callback=event_callback)
+
+If you want to add a "raw" event directly, you can use this example as a guid:
 
-Reports
-=======
+.. code-block:: python3
 
-Reporting is probably the most complicated of interactions within OpenADR. It involves the following steps:
+    from openleadr import OpenADRServer
+    from openleadr.objects import Event, EventDescriptor, EventSignal, Target, Interval
+    from datetime import datetime, timezone
+    from functools import partial
+
+    async def event_callback(ven_id, event_id, opt_status):
+        print(f"VEN {ven_id} responded {opt_status} to event {event_id}")
+
+    server = OpenADRServer(vtn_id='myvtn')
+    event = Event(event_descriptor=EventDescriptor(event_id='event001',
+                                                   modification_number=0,
+                                                   event_status='far',
+                                                   market_context='http://marketcontext01'),
+                  event_signals=[EventSignal(signal_id='signal001',
+                                             signal_type='level',
+                                             signal_name='simple',
+                                             intervals=[Interval(dtstart=now,
+                                                                 duration=datetime.timedelta(minutes=10),
+                                                                 signal_payload=1)]),
+                                 EventSignal(signal_id='signal002',
+                                             signal_type='price',
+                                             signal_name='ELECTRICITY_PRICE',
+                                             intervals=[Interval(dtstart=now,
+                                                                 duration=datetime.timedelta(minutes=10),
+                                                                 signal_payload=1)])],
+                  targets=[objects.Target(ven_id='ven123')])
+
+    server.add_raw_event(ven_id='ven123', event=event, callback=event_callback)
+
+If you want to add an event and wait for the response in a single coroutine, you can pass an asyncio Future instead of a function or coroutine as the callback argument:
 
-1. Party A makes its reporting capabilities known to party B using a :ref:`oadrRegisterReport` message.
-2. Party B responds with an :ref:`oadrRegisteredReport` message, optionally including an :ref:`oadrReportRequest` section that tells party A which party B is interested in.
-3. Party A reponds with an oadrCreatedReport message telling party B that it will periodically generate the reports.
+.. code-block:: python3
 
-This ceremony is performed once with the VTN as party A and once with the VEN as party A.
+    import asyncio
 
-The VEN party can collect the reports it requested from the VTN using either the :ref:`oadrPoll` or :ref:`oadrRequestReport` messages. The VTN will respond with an :ref:`oadrUpdateReport` message containing the actual report. The VEN should then respond with a :ref:`oadrUpdatedReport` message.
+    ...
 
-The VEN should actively supply the reports to the VTN using :ref:`oadrUpdateReport` messages, to which the VTN will respond with :ref:`oadrUpdatedReport` messages.
+    async def generate_event():
+        loop = asyncio.get_event_loop()
+        opt_status_future = loop.create_future()
+        server.add_event(..., callback=opt_status_future)
+        opt_status = await opt_status_future
+        print(f"The opt status for this event is {opt_status}")
 
-.. admonition:: Implementation Checklist
 
-    To benefit from the automatic reporting engine in OpenLEADR, you should implement the following items yourself:
+A word on event targets
+-----------------------
 
-    1. Configure the OpenADRServer() instance with your reporting capabilities and requirements
-    2. Implement a handlers that can retrieve the reports from your backend system
-    3. Implement a handler that deal with reports that come in from the clients
+The Target of your Event is an indication for the VEN which resources or devices should be affected. You can supply the target of the event in serveral ways:
 
+- Assigning the ``target`` parameter with a single ``objects.Target`` object.
+- Assigning the ``targets`` parameter with a list of ``objects.Target`` objects.
+- Assigning the ``targets_by_type`` parameters with a dict, that lists targets grouped by their type, like this:
 
-.. _server_implement:
+.. code-block:: python3
 
-Things you should implement
-===========================
+    server.add_event(...
+                     targets_by_type={'resource_id': ['resource01', 'resource02'],
+                                      'group_id': ['group01', 'group02']}
+                     )
 
-You should implement the following handlers:
+If you dont assign any Target, the target will be set to the ``ven_id`` that you specified.
 
-- ``on_poll(ven_id)``
-- ``on_request_event(ven_id)``
-- ``on_request_report(payload)``
-- ``on_create_party_registration(payload)``
 
-.. _server_meta:
+.. _server_reports:
 
-Non-OpenADR signals from the server
-===================================
+Reports
+=======
 
-The OpenLEADR Server can call the following handlers, which are not part of the regular openADR communication flow, but can help you develop a more robust event-driven system:
+Please see the :ref:`reporting` section.
 
-- ``on_ven_online(ven_id)``: called when a VEN sends an :ref:`oadrPoll`, :ref:`oadrRequestEvent` or :ref:`oadrRequestReport` message after it had been offline before.
-- ``on_ven_offline(ven_id)``: called when a VEN misses 3 consecutive poll intervals (configurable).
 
-Example implementation:
+.. _server_implement:
 
-.. code-block:: python3
+Things you should implement
+===========================
 
-    from openleadr import OpenADRServer
+You should implement the following handlers:
 
-    server = OpenADRServer(vtn_id='MyVTN')
-    server.add_handler('on_ven_online', on_ven_online)
-    server.add_handler('on_ven_offline', on_ven_offline)
+- ``on_create_party_registration(registration_info)``
+- ``on_register_report(ven_id, resource_id, measurement, unit, scale, min_sampling_interval, max_sampling_interval)``
 
-    async def on_ven_online(ven_id):
-        print(f"VEN {ven_id} is now online again!")
+Optionally:
 
-    async def on_ven_offline(ven_id):
-        print(f"VEN {ven_id} has gone AWOL")
+- ``on_poll(ven_id)``; only if you don't want to use the internal message queue.
 
 .. _server_signing_messages:
 
@@ -178,22 +310,6 @@ Message Handlers
 
 Your server has to deal with the different OpenADR messages. The way this works is that OpenLEADR will expose certain modules at the appropriate endpoints (like /oadrPoll and /EiRegister), and figure out what type of message is being sent. It will then call your handler with the contents of the message that are relevant for you to handle. This section provides an overview with examples for the different kinds of messages that you can expect and what should be returned.
 
-.. _server_on_created_event:
-
-on_created_event
-----------------
-
-The VEN informs you that they created an Event that you sent to them. You don't have to return anything.
-
-Return: `None`
-
-.. _server_on_request_event:
-
-on_request_event
-----------------
-
-The VEN is requesting the next Event that you have for it. You should return an Event. If you have no Events for this VEN, you should return `None`.
-
 .. _server_on_register_report:
 
 on_register_report
@@ -205,38 +321,58 @@ Signature:
 
 .. code-block:: python3
 
-    async def on_register_report(ven_id, reports):
+    async def on_register_report(ven_id, resource_id, measurement, unit, scale,
+                                 min_sampling_interval, max_sampling_interval):
+        # If we want this report:
+        return (callback, requested_sampling_interval)
+        # or
+        return None
 
+.. _server_on_query_registration:
 
-.. _server_on_created_report:
+on_query_registration
+---------------------
 
-on_created_report
------------------
+A prospective VEN is requesting information about your VTN, like the versions and transports you support. You should not implement this handler and let OpenLEADR handle this response.
 
-The VEN informs you that it created the automatic reporting that you requested. You don't have to return anything.
+.. _server_on_create_party_registration:
 
-Return: `None`
+on_create_party_registration
+----------------------------
 
-.. _server_on_update_report:
+The VEN tries to register with you. You will receive a registration_info dict that contains, among other things, a field `ven_name` which is how the VEN identifies itself. If the VEN is accepted, you return a ``ven_id, registration_id`` tuple. If not, return ``False``:
 
-on_update_report
-----------------
+.. code-block:: python3
 
-The VEN is sending you a fresh report with data. You don't have to return anything.
+    async def on_create_party_registration(registration_info):
+        ven_name = registration_info['ven_name']
+        ...
+        if ven_is_known:
+            return ven_id, registration_id
+        else
+            return None
 
-Signature:
+During this step, the VEN probably does not have a ``venID`` yet. If they connected using a secure TLS connection, the ``registration_info`` dict will contain the fingerprint of the public key that was used for this connection (``registration_info['fingerprint']``). Your ``on_create_party_registration`` handler should check this fingerprint value against a value that you received offline, to be sure that the ven with this venName is the correct VEN.
 
-.. code-block:: python3
+.. _server_on_cancel_party_registration:
+
+on_cancel_party_registration
+----------------------------
+
+The VEN informs you that they are cancelling their registration and no longer wish to be contacted by you.
+
+You should deregister the VEN internally, and return `None`.
+
+Return: ``None``
 
-    async def on_update_report(ven_id, report):
-        ...
-        return None
 
 .. _server_on_poll:
 
 on_poll
 -------
 
+You only need to implement this if you don't want to use the automatic internal message queue. If you add this handler to the server, the internal message queue will be automatically disabled.
+
 The VEN is requesting the next message that you have for it. You should return a tuple of message_type and message_payload as a dict. If there is no message for the VEN, you should return `None`.
 
 Signature:
@@ -247,36 +383,19 @@ Signature:
         ...
         return message_type, message_payload
 
-.. _server_on_query_registration:
-
-on_query_registration
----------------------
+If you implement your own on_poll handler, you should also include your own ``on_created_event`` handler that retrieves the opt status for a distributed event.
 
-A prospective VEN is requesting information about your VTN, like the versions and transports you support. You should not implement this handler and let OpenLEADR handle this response.
+.. _server_on_created_event:
 
-.. _server_on_create_party_registration:
+on_created_event
+----------------
 
-on_create_party_registration
-----------------------------
+You only need to implement this if you don't want to use the automatic internal message queue. Otherwise, you supply a per-event callback function when you add the event to the internal queue.
 
-The VEN tries to register with you. You will probably have manually configured the VEN beforehand, so you should look them up by their ven_name. You should have a ven_id that you generated ready.
-If they are allowed to register, return the ven_id (str), otherwise return False.
+Signature:
 
 .. code-block:: python3
 
-    async def on_create_party_registration(ven_name):
-        if ven_is_known:
-            return ven_id
-        else
-            return None
-
-.. _server_on_cancel_party_registration:
-
-on_cancel_party_registration
-----------------------------
-
-The VEN informs you that they are cancelling their registration and no longer wish to be contacted by you.
-
-You should deregister the VEN internally, and return `None`.
-
-Return: `None`
+    async def on_created_event(ven_id, event_id, opt_status):
+        print("Ven {ven_id} returned {opt_status} for event {event_id}")
+        # return None

openleadr/__init__.py

@@ -14,5 +14,26 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+# flake8: noqa
+
+import logging
 from .client import OpenADRClient
 from .server import OpenADRServer
+
+
+def enable_default_logging(level=logging.INFO):
+    """
+    Turn on logging to stdout.
+    :param level integer: The logging level you wish to use.
+                          Defaults to logging.INFO.
+    """
+    import sys
+    import logging
+    logger = logging.getLogger('openleadr')
+    handler_names = [handler.name for handler in logger.handlers]
+    if 'openleadr_default_handler' not in handler_names:
+        logger.setLevel(level)
+        logging_handler = logging.StreamHandler(stream=sys.stdout)
+        logging_handler.set_name('openleadr_default_handler')
+        logging_handler.setLevel(logging.DEBUG)
+        logger.addHandler(logging_handler)

openleadr/client.py

@@ -14,193 +14,361 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-"""
-OpenADR Client for Python
-"""
-
-import xmltodict
+import asyncio
+import inspect
+import logging
+import ssl
+import sys
 import random
-import aiohttp
-from openleadr.utils import peek, generate_id, certificate_fingerprint
-from openleadr.messaging import create_message, parse_message
-from openleadr import enums
 from datetime import datetime, timedelta, timezone
+from functools import partial
 from http import HTTPStatus
+
+import aiohttp
+from lxml.etree import XMLSyntaxError
+from signxml.exceptions import InvalidSignature
 from apscheduler.schedulers.asyncio import AsyncIOScheduler
-import asyncio
-from asyncio import iscoroutine
-from functools import partial
-import warnings
+from openleadr import enums, objects, errors
+from openleadr.messaging import create_message, parse_message, \
+                                validate_xml_schema, validate_xml_signature
+from openleadr import utils
+
+logger = logging.getLogger('openleadr')
 
-MEASURANDS = {'power_real': 'power_quantity',
-              'power_reactive': 'power_quantity',
-              'power_apparent': 'power_quantity',
-              'energy_real': 'energy_quantity',
-              'energy_reactive': 'energy_quantity',
-              'energy_active': 'energy_quantity'}
 
 class OpenADRClient:
     """
     Main client class. Most of these methods will be called automatically, but
     you can always choose to call them manually.
     """
-    def __init__(self, ven_name, vtn_url, debug=False, cert=None, key=None, passphrase=None, vtn_fingerprint=None):
+    def __init__(self, ven_name, vtn_url, debug=False, cert=None, key=None,
+                 passphrase=None, vtn_fingerprint=None, show_fingerprint=True, ca_file=None,
+                 allow_jitter=True):
         """
         Initializes a new OpenADR Client (Virtual End Node)
 
         :param str ven_name: The name for this VEN
         :param str vtn_url: The URL of the VTN (Server) to connect to
         :param bool debug: Whether or not to print debugging messages
-        :param str cert: The path to a PEM-formatted Certificate file to use for signing messages
-        :param str key: The path to a PEM-formatted Private Key file to use for signing messages
-        :param str fingerprint: The fingerprint for the VTN's certificate to verify incomnig messages
+        :param str cert: The path to a PEM-formatted Certificate file to use
+                         for signing messages.
+        :param str key: The path to a PEM-formatted Private Key file to use
+                        for signing messages.
+        :param str fingerprint: The fingerprint for the VTN's certificate to
+                                verify incomnig messages
+        :param str show_fingerprint: Whether to print your own fingerprint
+                                     on startup. Defaults to True.
+        :param str ca_file: The path to the PEM-formatted CA file for validating the VTN server's
+                            certificate.
         """
 
         self.ven_name = ven_name
+        if vtn_url.endswith("/"):
+            vtn_url = vtn_url[:-1]
         self.vtn_url = vtn_url
         self.ven_id = None
+        self.registration_id = None
         self.poll_frequency = None
+        self.vtn_fingerprint = vtn_fingerprint
         self.debug = debug
-        self.reports = {}           # Mapping of all available reports from the VEN
-        self.report_requests = {}   # Mapping of the reports requested by the VTN
-        self.report_schedulers = {} # Mapping between reportRequestIDs and our internal report schedulers
+
+        self.reports = []
+        self.report_callbacks = {}              # Holds the callbacks for each specific report
+        self.report_requests = []               # Keep track of the report requests from the VTN
+        self.incomplete_reports = {}            # Holds reports that are being populated over time
+        self.pending_reports = asyncio.Queue()  # Holds reports that are waiting to be sent
         self.scheduler = AsyncIOScheduler()
-        self.client_session = aiohttp.ClientSession()
+        self.client_session = None
+        self.report_queue_task = None
+
+        self.received_events = {}               # Holds the events that we received.
+        self.responded_events = {}              # Holds the events that we already saw.
+
+        self.cert_path = cert
+        self.key_path = key
+        self.passphrase = passphrase
+        self.ca_file = ca_file
+        self.allow_jitter = allow_jitter
 
         if cert and key:
             with open(cert, 'rb') as file:
                 cert = file.read()
             with open(key, 'rb') as file:
                 key = file.read()
-            print("*" * 80)
-            print("Your VEN Certificate Fingerprint is", certificate_fingerprint(cert))
-            print("Please deliver this fingerprint to the VTN you are connecting to.")
-            print("You do not need to keep this a secret.")
-            print("*" * 80)
+            if show_fingerprint:
+                print("")
+                print("*" * 80)
+                print("Your VEN Certificate Fingerprint is ".center(80))
+                print(f"{utils.certificate_fingerprint(cert).center(80)}".center(80))
+                print("Please deliver this fingerprint to the VTN.".center(80))
+                print("You do not need to keep this a secret.".center(80))
+                print("*" * 80)
+                print("")
 
         self._create_message = partial(create_message,
                                        cert=cert,
                                        key=key,
                                        passphrase=passphrase)
-        self._parse_message = partial(parse_message,
-                                      fingerprint=vtn_fingerprint)
-
 
     async def run(self):
         """
         Run the client in full-auto mode.
         """
-        if not hasattr(self, 'on_event'):
-            raise NotImplementedError("You must implement an on_event function or coroutine.")
-
+        # if not hasattr(self, 'on_event'):
+        #     raise NotImplementedError("You must implement on_event.")
+        self.loop = asyncio.get_event_loop()
         await self.create_party_registration()
 
         if not self.ven_id:
-            print("No VEN ID received from the VTN, aborting registration.")
+            logger.error("No VEN ID received from the VTN, aborting.")
+            await self.stop()
             return
 
         if self.reports:
-            await self.register_report()
+            await self.register_reports(self.reports)
+            self.report_queue_task = self.loop.create_task(self._report_queue_worker())
+
+        await self._poll()
 
         # Set up automatic polling
-        if self.poll_frequency.total_seconds() < 60:
-            cron_second = f"*/{self.poll_frequency.seconds}"
-            cron_minute = "*"
-            cron_hour = "*"
-        elif self.poll_frequency.total_seconds() < 3600:
-            cron_second = "0"
-            cron_minute = f'*/{int(self.poll_frequency.total_seconds() / 60)}'
-            cron_hour = "*"
-        elif self.poll_frequency.total_seconds() < 86400:
-            cron_second = "0"
-            cron_minute = "0"
-            cron_hour = f'*/{int(self.poll_frequency.total_seconds() / 3600)}'
-        elif self.poll_frequency.total_seconds() > 86400:
-            print("Polling with intervals of more than 24 hours is not supported.")
+        if self.poll_frequency > timedelta(hours=24):
+            logger.warning("Polling with intervals of more than 24 hours is not supported. "
+                           "Will use 24 hours as the logging interval.")
+            self.poll_frequency = timedelta(hours=24)
+        cron_config = utils.cron_config(self.poll_frequency, randomize_seconds=self.allow_jitter)
+
+        self.scheduler.add_job(self._poll,
+                               trigger='cron',
+                               **cron_config)
+        self.scheduler.add_job(self._event_cleanup,
+                               trigger='interval',
+                               seconds=300)
+        self.scheduler.start()
+
+    async def stop(self):
+        """
+        Cleanly stops the client. Run this coroutine before closing your event loop.
+        """
+        if self.scheduler.running:
+            self.scheduler.shutdown()
+        if self.report_queue_task:
+            self.report_queue_task.cancel()
+        if sys.version_info.minor > 8:
+            delayed_call_tasks = [task for task in asyncio.all_tasks() if task.get_name().startswith('DelayedCall')]
+            for task in delayed_call_tasks:
+                task.cancel()
+        await self.client_session.close()
+        await asyncio.sleep(0)
+
+    def add_handler(self, handler, callback):
+        """
+        Add a callback for the given situation
+        """
+        if handler not in ('on_event', 'on_update_event'):
+            logger.error("'handler' must be either on_event or on_update_event")
             return
 
-        self.scheduler.add_job(self._poll, trigger='cron', second=cron_second, minute=cron_minute, hour=cron_hour)
-        self.scheduler.start()
+        setattr(self, handler, callback)
 
-    def add_report(self, callable, report_id, report_name, reading_type, report_type,
-                         sampling_rate, resource_id, measurand, unit, scale="none",
-                         power_ac=True, power_hertz=50, power_voltage=230, market_context=None):
+    def add_report(self, callback, resource_id, measurement=None,
+                   data_collection_mode='incremental',
+                   report_specifier_id=None, r_id=None,
+                   report_name=enums.REPORT_NAME.TELEMETRY_USAGE,
+                   reading_type=enums.READING_TYPE.DIRECT_READ,
+                   report_type=enums.REPORT_TYPE.READING, sampling_rate=None, data_source=None,
+                   scale="none", unit=None, power_ac=True, power_hertz=50, power_voltage=230,
+                   market_context=None, end_device_asset_mrid=None, report_data_source=None):
         """
         Add a new reporting capability to the client.
 
-        :param callable callable: A callable or coroutine that will fetch the value for a specific report. This callable will be passed the report_id and the r_id of the requested value.
-        :param str report_id: A unique identifier for this report.
+        :param callable callback: A callback or coroutine that will fetch the value for a specific
+                                  report. This callback will be passed the report_id and the r_id
+                                  of the requested value.
+        :param str resource_id: A specific name for this resource within this report.
+        :param str measurement: The quantity that is being measured (openleadr.enums.MEASUREMENTS).
+                                Optional for TELEMETRY_STATUS reports.
+        :param str data_collection_mode: Whether you want the data to be collected incrementally
+                                         or at once. If the VTN requests the sampling interval to be
+                                         higher than the reporting interval, this setting determines
+                                         if the callback should be called at the sampling rate (with
+                                         no args, assuming it returns the current value), or at the
+                                         reporting interval (with date_from and date_to as keyword
+                                         arguments). Choose 'incremental' for the former case, or
+                                         'full' for the latter case.
+        :param str report_specifier_id: A unique identifier for this report. Leave this blank for a
+                                        random generated id, or fill it in if your VTN depends on
+                                        this being a known value, or if it needs to be constant
+                                        between restarts of the client.
+        :param str r_id: A unique identifier for a datapoint in a report. The same remarks apply as
+                         for the report_specifier_id.
         :param str report_name: An OpenADR name for this report (one of openleadr.enums.REPORT_NAME)
         :param str reading_type: An OpenADR reading type (found in openleadr.enums.READING_TYPE)
         :param str report_type: An OpenADR report type (found in openleadr.enums.REPORT_TYPE)
         :param datetime.timedelta sampling_rate: The sampling rate for the measurement.
-        :param resource_id: A specific name for this resource within this report.
         :param str unit: The unit for this measurement.
-
+        :param boolean power_ac: Whether the power is AC (True) or DC (False).
+                                 Only required when supplying a power-related measurement.
+        :param int power_hertz: Grid frequency of the power.
+                                Only required when supplying a power-related measurement.
+        :param int power_voltage: Voltage of the power.
+                                  Only required when supplying a power-related measurement.
+        :param str market_context: The Market Context that this report belongs to.
+        :param str end_device_asset_mrid: the Meter ID for the end device that is measured by this report.
+        :param report_data_source: A (list of) target(s) that this report is related to.
         """
 
-        if report_name not in enums.REPORT_NAME.values:
-            raise ValueError(f"{report_name} is not a valid report_name. Valid options are {', '.join(enums.REPORT_NAME.values)}.")
-        if reading_type not in enums.READING_TYPE.values:
-            raise ValueError(f"{reading_type} is not a valid reading_type. Valid options are {', '.join(enums.READING_TYPE.values)}.")
-        if report_type not in enums.REPORT_TYPE.values:
-            raise ValueError(f"{report_type} is not a valid report_type. Valid options are {', '.join(enums.REPORT_TYPE.values)}.")
-        if measurand not in MEASURANDS:
-            raise ValueError(f"{measurand} is not a valid measurand. Valid options are 'power_real', 'power_reactive', 'power_apparent', 'energy_real', 'energy_reactive', 'energy_active', 'energy_quantity', 'voltage'")
+        # Verify input
+        if report_name not in enums.REPORT_NAME.values and not report_name.startswith('x-'):
+            raise ValueError(f"{report_name} is not a valid report_name. Valid options are "
+                             f"{', '.join(enums.REPORT_NAME.values)}",
+                             " or any name starting with 'x-'.")
+        if reading_type not in enums.READING_TYPE.values and not reading_type.startswith('x-'):
+            raise ValueError(f"{reading_type} is not a valid reading_type. Valid options are "
+                             f"{', '.join(enums.READING_TYPE.values)}"
+                             " or any name starting with 'x-'.")
+        if report_type not in enums.REPORT_TYPE.values and not report_type.startswith('x-'):
+            raise ValueError(f"{report_type} is not a valid report_type. Valid options are "
+                             f"{', '.join(enums.REPORT_TYPE.values)}"
+                             " or any name starting with 'x-'.")
         if scale not in enums.SI_SCALE_CODE.values:
-            raise ValueError(f"{scale} is not a valid scale. Valid options are {', '.join(enums.SI_SCALE_CODE.values)}")
-
-        report_description = {'market_context': market_context,
-                              'r_id': resource_id,
-                              'reading_type': reading_type,
-                              'report_type': report_type,
-                              'sampling_rate': {'max_period': sampling_rate,
-                                                'min_period': sampling_rate,
-                                                'on_change': False},
-                               measurand: {'item_description': measurand,
-                                           'item_units': unit,
-                                           'si_scale_code': scale}}
-        if 'power' in measurand:
-            report_description[measurand]['power_attributes'] = {'ac': power_ac, 'hertz': power_hertz, 'voltage': power_voltage}
-
-        if report_id in self.reports:
-            report = self.reports[report_id]['report_descriptions'].append(report_description)
+            raise ValueError(f"{scale} is not a valid scale. Valid options are "
+                             f"{', '.join(enums.SI_SCALE_CODE.values)}")
+
+        if sampling_rate is None:
+            sampling_rate = objects.SamplingRate(min_period=timedelta(seconds=10),
+                                                 max_period=timedelta(hours=24),
+                                                 on_change=False)
+        elif isinstance(sampling_rate, timedelta):
+            sampling_rate = objects.SamplingRate(min_period=sampling_rate,
+                                                 max_period=sampling_rate,
+                                                 on_change=False)
+
+        if data_collection_mode not in ('incremental', 'full'):
+            raise ValueError("The data_collection_mode should be 'incremental' or 'full'.")
+
+        if data_collection_mode == 'full':
+            args = inspect.signature(callback).parameters
+            if not ('date_from' in args and 'date_to' in args and 'sampling_interval' in args):
+                raise TypeError("Your callback function must accept the 'date_from', 'date_to' "
+                                "and 'sampling_interval' arguments if used "
+                                "with data_collection_mode 'full'.")
+
+        # Determine the correct item name, item description and unit
+        if report_name == 'TELEMETRY_STATUS':
+            item_base = None
+        elif isinstance(measurement, objects.Measurement):
+            item_base = measurement
+        elif isinstance(measurement, dict):
+            utils.validate_report_measurement_dict(measurement)
+            power_attributes = object.PowerAttributes(**measurement.get('power_attributes')) or None
+            item_base = objects.Measurement(name=measurement['name'],
+                                            description=measurement['description'],
+                                            unit=measurement['unit'],
+                                            scale=measurement.get('scale'),
+                                            power_attributes=power_attributes)
+        elif measurement.upper() in enums.MEASUREMENTS.members:
+            item_base = enums.MEASUREMENTS[measurement.upper()]
         else:
-            report = {'callable': callable,
-                      'created_date_time': datetime.now(timezone.utc),
-                      'report_id': report_id,
-                      'report_name': report_name,
-                      'report_request_id': generate_id(),
-                      'report_specifier_id': report_id + "_" + report_name.lower(),
-                      'report_descriptions': [report_description]}
-        self.reports[report_id] = report
-        self.report_ids[resource_id] = {'item_base': measurand}
+            item_base = objects.Measurement(name='customUnit',
+                                            description=measurement,
+                                            unit=unit,
+                                            scale=scale)
+
+        if report_name != 'TELEMETRY_STATUS' and scale is not None:
+            if item_base.scale is not None:
+                if scale in enums.SI_SCALE_CODE.values:
+                    item_base.scale = scale
+            else:
+                raise ValueError("The 'scale' argument must be one of '{'. ',join(enums.SI_SCALE_CODE.values)}")
+
+        # Check if unit is compatible
+        if unit is not None and unit != item_base.unit and unit not in item_base.acceptable_units:
+            logger.warning(f"The supplied unit {unit} for measurement {measurement} "
+                           f"will be ignored, {item_base.unit} will be used instead. "
+                           f"Allowed units for this measurement are: "
+                           f"{', '.join(item_base.acceptable_units)}")
+
+        # Get or create the relevant Report
+        if report_specifier_id:
+            report = utils.find_by(self.reports,
+                                   'report_name', report_name,
+                                   'report_specifier_id', report_specifier_id)
+        else:
+            report = utils.find_by(self.reports, 'report_name', report_name)
+
+        if not report:
+            report_specifier_id = report_specifier_id or utils.generate_id()
+            report = objects.Report(created_date_time=datetime.now(),
+                                    report_name=report_name,
+                                    report_specifier_id=report_specifier_id,
+                                    data_collection_mode=data_collection_mode)
+            self.reports.append(report)
+
+        # Add the new report description to the report
+        target = objects.Target(resource_id=resource_id)
+        r_id = utils.generate_id()
+        report_description = objects.ReportDescription(r_id=r_id,
+                                                       reading_type=reading_type,
+                                                       report_data_source=target,
+                                                       report_subject=target,
+                                                       report_type=report_type,
+                                                       sampling_rate=sampling_rate,
+                                                       measurement=item_base,
+                                                       market_context=market_context)
+        self.report_callbacks[(report.report_specifier_id, r_id)] = callback
+        report.report_descriptions.append(report_description)
+        return report_specifier_id, r_id
+
+    ###########################################################################
+    #                                                                         #
+    #                             POLLING METHODS                             #
+    #                                                                         #
+    ###########################################################################
+
+    async def poll(self):
+        """
+        Request the next available message from the Server. This coroutine is called automatically.
+        """
+        service = 'OadrPoll'
+        message = self._create_message('oadrPoll', ven_id=self.ven_id)
+        response_type, response_payload = await self._perform_request(service, message)
+        return response_type, response_payload
+
+    ###########################################################################
+    #                                                                         #
+    #                         REGISTRATION METHODS                            #
+    #                                                                         #
+    ###########################################################################
 
     async def query_registration(self):
         """
         Request information about the VTN.
         """
-        request_id = generate_id()
+        request_id = utils.generate_id()
         service = 'EiRegisterParty'
         message = self._create_message('oadrQueryRegistration', request_id=request_id)
         response_type, response_payload = await self._perform_request(service, message)
         return response_type, response_payload
 
     async def create_party_registration(self, http_pull_model=True, xml_signature=False,
-                                  report_only=False, profile_name='2.0b',
-                                  transport_name='simpleHttp', transport_address=None, ven_id=None):
+                                        report_only=False, profile_name='2.0b',
+                                        transport_name='simpleHttp', transport_address=None,
+                                        ven_id=None):
         """
         Take the neccessary steps to register this client with the server.
 
         :param bool http_pull_model: Whether to use the 'pull' model for HTTP.
         :param bool xml_signature: Whether to sign each XML message.
-        :param bool report_only: Whether or not this is a reporting-only client which does not deal with Events.
+        :param bool report_only: Whether or not this is a reporting-only client
+                                 which does not deal with Events.
         :param str profile_name: Which OpenADR profile to use.
         :param str transport_name: The transport name to use. Either 'simpleHttp' or 'xmpp'.
-        :param str transport_address: Which public-facing address the server should use to communicate.
-        :param str ven_id: The ID for this VEN. If you leave this blank, a VEN_ID will be assigned by the VTN.
+        :param str transport_address: Which public-facing address the server should use
+                                      to communicate.
+        :param str ven_id: The ID for this VEN. If you leave this blank,
+                           a VEN_ID will be assigned by the VTN.
         """
-        request_id = generate_id()
+        request_id = utils.generate_id()
         service = 'EiRegisterParty'
         payload = {'ven_name': self.ven_name,
                    'http_pull_model': http_pull_model,
@@ -211,29 +379,40 @@ class OpenADRClient:
                    'transport_address': transport_address}
         if ven_id:
             payload['ven_id'] = ven_id
-        message = self._create_message('oadrCreatePartyRegistration', request_id=generate_id(), **payload)
+        message = self._create_message('oadrCreatePartyRegistration',
+                                       request_id=request_id,
+                                       **payload)
         response_type, response_payload = await self._perform_request(service, message)
         if response_type is None:
             return
         if response_payload['response']['response_code'] != 200:
             status_code = response_payload['response']['response_code']
             status_description = response_payload['response']['response_description']
-            print(f"Got error on Create Party Registration: {status_code} {status_description}")
+            logger.error(f"Got error on Create Party Registration: "
+                         f"{status_code} {status_description}")
             return
         self.ven_id = response_payload['ven_id']
-        self.poll_frequency = response_payload.get('requested_oadr_poll_freq', timedelta(seconds=10))
-        print(f"VEN is now registered with ID {self.ven_id}")
-        print(f"The polling frequency is {self.poll_frequency}")
+        self.registration_id = response_payload['registration_id']
+        self.poll_frequency = response_payload.get('requested_oadr_poll_freq',
+                                                   timedelta(seconds=10))
+        logger.info(f"VEN is now registered with ID {self.ven_id}")
+        logger.info(f"The polling frequency is {self.poll_frequency}")
         return response_type, response_payload
 
     async def cancel_party_registration(self):
         raise NotImplementedError("Cancel Registration is not yet implemented")
 
+    ###########################################################################
+    #                                                                         #
+    #                              EVENT METHODS                              #
+    #                                                                         #
+    ###########################################################################
+
     async def request_event(self, reply_limit=1):
         """
         Request the next Event from the VTN, if it has any.
         """
-        payload = {'request_id': generate_id(),
+        payload = {'request_id': utils.generate_id(),
                    'ven_id': self.ven_id,
                    'reply_limit': reply_limit}
         message = self._create_message('oadrRequestEvent', **payload)
@@ -259,133 +438,429 @@ class OpenADRClient:
         message = self._create_message('oadrCreatedEvent', **payload)
         response_type, response_payload = await self._perform_request(service, message)
 
-    async def register_report(self):
+    ###########################################################################
+    #                                                                         #
+    #                             REPORTING METHODS                           #
+    #                                                                         #
+    ###########################################################################
+
+    async def register_reports(self, reports):
         """
-        Tell the VTN about our reporting capabilities.
+        Tell the VTN about our reports. The VTN miht respond with an
+        oadrCreateReport message that tells us which reports are to be sent.
         """
-        request_id = generate_id()
-
-        payload = {'request_id': generate_id(),
+        request_id = utils.generate_id()
+        payload = {'request_id': request_id,
                    'ven_id': self.ven_id,
-                   'reports': self.reports}
+                   'reports': reports,
+                   'report_request_id': 0}
 
         service = 'EiReport'
         message = self._create_message('oadrRegisterReport', **payload)
         response_type, response_payload = await self._perform_request(service, message)
 
-        # Remember which reports the VTN is interested in
+        # Handle the subscriptions that the VTN is interested in.
+        if 'report_requests' in response_payload:
+            for report_request in response_payload['report_requests']:
+                await self.create_report(report_request)
 
-        return response_type, response_payload
+        message_type = 'oadrCreatedReport'
+        message_payload = {}
 
-    async def created_report(self):
-        pass
+        return message_type, message_payload
 
-    async def poll(self):
+    async def create_report(self, report_request):
         """
-        Request the next available message from the Server. This coroutine is called automatically.
+        Add the requested reports to the reporting mechanism.
+        This is called when the VTN requests reports from us.
+
+        :param report_request dict: The oadrReportRequest dict from the VTN.
+        """
+        # Get the relevant variables from the report requests
+        report_request_id = report_request['report_request_id']
+        report_specifier_id = report_request['report_specifier']['report_specifier_id']
+        report_back_duration = report_request['report_specifier'].get('report_back_duration')
+        granularity = report_request['report_specifier']['granularity']
+
+        # Check if this report actually exists
+        report = utils.find_by(self.reports, 'report_specifier_id', report_specifier_id)
+        if not report:
+            logger.error(f"A non-existant report with report_specifier_id "
+                         f"{report_specifier_id} was requested.")
+            return False
+
+        # Check and collect the requested r_ids for this report
+        requested_r_ids = []
+        for specifier_payload in report_request['report_specifier']['specifier_payloads']:
+            r_id = specifier_payload['r_id']
+            # Check if the requested r_id actually exists
+            rd = utils.find_by(report.report_descriptions, 'r_id', r_id)
+            if not rd:
+                logger.error(f"A non-existant report with r_id {r_id} "
+                             f"inside report with report_specifier_id {report_specifier_id} "
+                             f"was requested.")
+                continue
+
+            # Check if the requested measurement exists and if the correct unit is requested
+            if 'measurement' in specifier_payload:
+                measurement = specifier_payload['measurement']
+                if measurement['description'] != rd.measurement.description:
+                    logger.error(f"A non-matching measurement description for report with "
+                                 f"report_request_id {report_request_id} and r_id {r_id} was given "
+                                 f"by the VTN. Offered: {rd.measurement.description}, "
+                                 f"requested: {measurement['description']}")
+                    continue
+                if measurement['unit'] != rd.measurement.unit:
+                    logger.error(f"A non-matching measurement unit for report with "
+                                 f"report_request_id {report_request_id} and r_id {r_id} was given "
+                                 f"by the VTN. Offered: {rd.measurement.unit}, "
+                                 f"requested: {measurement['unit']}")
+                    continue
+
+            if granularity is not None:
+                if not rd.sampling_rate.min_period <= granularity <= rd.sampling_rate.max_period:
+                    logger.error(f"An invalid sampling rate {granularity} was requested for report "
+                                 f"with report_specifier_id {report_specifier_id} and r_id {r_id}. "
+                                 f"The offered sampling rate was between "
+                                 f"{rd.sampling_rate.min_period} and "
+                                 f"{rd.sampling_rate.max_period}")
+                    continue
+            else:
+                # If no granularity is specified, set it to the lowest sampling rate.
+                granularity = rd.sampling_rate.max_period
+
+            requested_r_ids.append(r_id)
+
+        callback = partial(self.update_report, report_request_id=report_request_id)
+
+        reporting_interval = report_back_duration or granularity
+        job = self.scheduler.add_job(func=callback,
+                                     trigger='cron',
+                                     **utils.cron_config(reporting_interval))
+
+        self.report_requests.append({'report_request_id': report_request_id,
+                                     'report_specifier_id': report_specifier_id,
+                                     'report_back_duration': report_back_duration,
+                                     'r_ids': requested_r_ids,
+                                     'granularity': granularity,
+                                     'job': job})
+
+    async def create_single_report(self, report_request):
+        """
+        Create a single report in response to a request from the VTN.
         """
-        service = 'OadrPoll'
-        message = self._create_message('oadrPoll', ven_id=self.ven_id)
-        response_type, response_payload = await self._perform_request(service, message)
-        return response_type, response_payload
 
-    async def update_report(self, report_id, resource_id=None):
+    async def update_report(self, report_request_id):
         """
-        Calls the previously registered report callable, and send the result as a message to the VTN.
+        Call the previously registered report callback and send the result as a message to the VTN.
         """
-        if not resource_id:
-            resource_ids = self.reports[report_id]['report_descriptions'].keys()
-        elif isinstance(resource_id, str):
-            resource_ids = [resource_id]
+        logger.debug(f"Running update_report for {report_request_id}")
+        report_request = utils.find_by(self.report_requests, 'report_request_id', report_request_id)
+        granularity = report_request['granularity']
+        report_back_duration = report_request['report_back_duration']
+        report_specifier_id = report_request['report_specifier_id']
+        report = utils.find_by(self.reports, 'report_specifier_id', report_specifier_id)
+        data_collection_mode = report.data_collection_mode
+
+        if report_request_id in self.incomplete_reports:
+            logger.debug("We were already compiling this report")
+            outgoing_report = self.incomplete_reports[report_request_id]
         else:
-            resource_ids = resource_id
-        value = self.reports[report_id]['callable'](resource_id)
-        if iscoroutine(value):
-            value = await value
-
-        report_type = self.reports[report_id][resource_id]['report_type']
-        for measurand in MEASURAND:
-            if measurand in self.reports[report_id][resource_id]:
-                item_base = measurand
-                break
-        report = {'report_id': report_id,
-                  'report_descriptions': {resource_id: {MEASURANDS[measurand]: {'quantity': value,
-                                                                         measurand: self.reports[report_id][resource_id][measurand]},
-                                          'report_type': self.reports[report_id][resource_id]['report_type'],
-                                          'reading_type': self.reports[report_id][resource_id]['reading_type']}},
-                  'report_name': self.report['report_id']['report_name'],
-                  'report_request_id': self.reports['report_id']['report_request_id'],
-                  'report_specifier_id': self.report['report_id']['report_specifier_id'],
-                  'created_date_time': datetime.now(timezone.utc)}
+            logger.debug("There is no report in progress")
+            outgoing_report = objects.Report(report_request_id=report_request_id,
+                                             report_specifier_id=report.report_specifier_id,
+                                             report_name=report.report_name,
+                                             intervals=[])
+
+        intervals = outgoing_report.intervals or []
+        if data_collection_mode == 'full':
+            if report_back_duration is None:
+                report_back_duration = granularity
+            date_to = datetime.now(timezone.utc)
+            date_from = date_to - max(report_back_duration, granularity)
+            for r_id in report_request['r_ids']:
+                report_callback = self.report_callbacks[(report_specifier_id, r_id)]
+                result = report_callback(date_from=date_from,
+                                         date_to=date_to,
+                                         sampling_interval=granularity)
+                if asyncio.iscoroutine(result):
+                    result = await result
+                for dt, value in result:
+                    report_payload = objects.ReportPayload(r_id=r_id, value=value)
+                    intervals.append(objects.ReportInterval(dtstart=dt,
+                                                            report_payload=report_payload))
 
-        service = 'EiReport'
-        message = self._create_message('oadrUpdateReport', report)
-        response_type, response_payload = self._perform_request(service, message)
-        if response_type is not None:
-            # We might get a oadrCancelReport message in this thing:
-            if 'cancel_report' in response_payload:
-                print("TODO: cancel this report")
+        else:
+            for r_id in report_request['r_ids']:
+                report_callback = self.report_callbacks[(report_specifier_id, r_id)]
+                result = report_callback()
+                if asyncio.iscoroutine(result):
+                    result = await result
+                if isinstance(result, (int, float)):
+                    result = [(datetime.now(timezone.utc), result)]
+                for dt, value in result:
+                    logger.info(f"Adding {dt}, {value} to report")
+                    report_payload = objects.ReportPayload(r_id=r_id, value=value)
+                    intervals.append(objects.ReportInterval(dtstart=dt,
+                                                            report_payload=report_payload))
+        outgoing_report.intervals = intervals
+        logger.info(f"The number of intervals in the report is now {len(outgoing_report.intervals)}")
+
+        # Figure out if the report is complete after this sampling
+        if data_collection_mode == 'incremental' and report_back_duration is not None\
+                and report_back_duration > granularity:
+            report_interval = report_back_duration.total_seconds()
+            sampling_interval = granularity.total_seconds()
+            expected_len = len(report_request['r_ids']) * int(report_interval / sampling_interval)
+            if len(outgoing_report.intervals) == expected_len:
+                logger.info("The report is now complete with all the values. Will queue for sending.")
+                if self.allow_jitter:
+                    delay = random.uniform(0, min(30, report_interval / 2))
+                    if sys.version_info.minor >= 8:
+                        name = {'name': f'DelayedCall-OutgoingReport-{utils.generate_id()}'}
+                    else:
+                        name = {}
+                    self.loop.create_task(utils.delayed_call(func=self.pending_reports.put(outgoing_report),
+                                                             delay=delay), **name)
+                else:
+                    await self.pending_reports.put(self.incomplete_reports.pop(report_request_id))
+            else:
+                logger.debug("The report is not yet complete, will hold until it is.")
+                self.incomplete_reports[report_request_id] = outgoing_report
+        else:
+            logger.info("Report will be sent now.")
+            if self.allow_jitter:
+                delay = random.uniform(0, min(30, granularity.total_seconds() / 2))
+                if sys.version_info.minor >= 8:
+                    name = {'name': f'DelayedCall-OutgoingReport-{utils.generate_id()}'}
+                else:
+                    name = {}
+                self.loop.create_task(utils.delayed_call(func=self.pending_reports.put(outgoing_report),
+                                                         delay=delay), **name)
+            else:
+                await self.pending_reports.put(outgoing_report)
+
+    async def cancel_report(self, payload):
+        """
+        Cancel this report.
+        """
+
+    async def _report_queue_worker(self):
+        """
+        A Queue worker that pushes out the pending reports.
+        """
 
+        while True:
+            report = await self.pending_reports.get()
+            service = 'EiReport'
+            message = self._create_message('oadrUpdateReport', reports=[report])
+
+            try:
+                response_type, response_payload = await self._perform_request(service, message)
+            except Exception as err:
+                logger.error(f"Unable to send the report to the VTN. Error: {err}")
+            else:
+                if 'cancel_report' in response_payload:
+                    await self.cancel_report(response_payload['cancel_report'])
+
+    ###########################################################################
+    #                                                                         #
+    #                                  PLACEHOLDER                            #
+    #                                                                         #
+    ###########################################################################
+
+    async def on_event(self, event):
+        """
+        Placeholder for the on_event handler.
+        """
+        logger.warning("You should implement your own on_event handler. This handler receives "
+                       "an Event dict and should return either 'optIn' or 'optOut' based on your "
+                       "choice. Will opt out of the event for now.")
+        return 'optOut'
+
+    async def on_update_event(self, event):
+        """
+        Placeholder for the on_update_event handler.
+        """
+        logger.warning("You should implement your own on_update_event handler. This handler receives "
+                       "an Event dict and should return either 'optIn' or 'optOut' based on your "
+                       "choice. Will re-use the previous opt status for this event_id for now")
+        if event['event_descriptor']['event_id'] in self.events:
+            return self.responded_events['event_id']
+
+    async def on_register_report(self, report):
+        """
+        Placeholder for the on_register_report handler.
+        """
+
+    ###########################################################################
+    #                                                                         #
+    #                                  LOW LEVEL                              #
+    #                                                                         #
+    ###########################################################################
 
     async def _perform_request(self, service, message):
-        if self.debug:
-            print(f"Client is sending {message}")
+        await self._ensure_client_session()
+        logger.debug(f"Client is sending {message}")
         url = f"{self.vtn_url}/{service}"
         try:
             async with self.client_session.post(url, data=message) as req:
+                content = await req.read()
                 if req.status != HTTPStatus.OK:
-                    warnings.warn(f"Non-OK status when performing a request to {url} with data {message}: {req.status}")
+                    logger.warning(f"Non-OK status {req.status} when performing a request to {url} "
+                                   f"with data {message}: {req.status} {content.decode('utf-8')}")
                     return None, {}
-                content = await req.read()
-                if self.debug:
-                    print(content.decode('utf-8'))
-        except:
+                logger.debug(content.decode('utf-8'))
+        except aiohttp.client_exceptions.ClientConnectorError as err:
             # Could not connect to server
-            warnings.warn(f"Could not connect to server with URL {self.vtn_url}")
+            logger.error(f"Could not connect to server with URL {self.vtn_url}:")
+            logger.error(f"{err.__class__.__name__}: {str(err)}")
+            return None, {}
+        except Exception as err:
+            logger.error(f"Request error {err.__class__.__name__}:{err}")
             return None, {}
         try:
-            message_type, message_payload = self._parse_message(content)
+            tree = validate_xml_schema(content)
+            if self.vtn_fingerprint:
+                validate_xml_signature(tree)
+            message_type, message_payload = parse_message(content)
+        except XMLSyntaxError as err:
+            logger.warning(f"Incoming message did not pass XML schema validation: {err}")
+            return None, {}
+        except errors.FingerprintMismatch as err:
+            logger.warning(err)
+            return None, {}
+        except InvalidSignature:
+            logger.warning("Incoming message had invalid signature, ignoring.")
+            return None, {}
         except Exception as err:
-            warnings.warn(f"The incoming message could not be parsed or validated: {content}.")
-            raise err
+            logger.error(f"The incoming message could not be parsed or validated: {err}")
             return None, {}
+        if 'response' in message_payload and 'response_code' in message_payload['response']:
+            if message_payload['response']['response_code'] != 200:
+                logger.warning("We got a non-OK OpenADR response from the server: "
+                               f"{message_payload['response']['response_code']}: "
+                               f"{message_payload['response']['response_description']}")
         return message_type, message_payload
 
     async def _on_event(self, message):
-        if self.debug:
-            print("ON_EVENT")
-        result = self.on_event(message)
-        if iscoroutine(result):
-            result = await result
-
-        if self.debug:
-            print(f"Now responding with {result}")
-        request_id = message['request_id']
-        event_id = message['events'][0]['event_descriptor']['event_id']
-        await self.created_event(request_id, event_id, result)
-        return
+        logger.debug("The VEN received an event")
+        events = message['events']
+        try:
+            results = []
+            for event in message['events']:
+                event_id = event['event_descriptor']['event_id']
+                event_status = event['event_descriptor']['event_status']
+                modification_number = event['event_descriptor']['modification_number']
+                if event_id in self.received_events:
+                    if self.received_events[event_id]['event_descriptor']['modification_number'] == modification_number:
+                        # Re-submit the same opt type as we already had previously
+                        result = self.responded_events[event_id]
+                    else:
+                        # Wait for the result of the on_update_event handler
+                        result = self.on_update_event(event)
+                else:
+                    # Wait for the result of the on_event
+                    self.received_events[event_id] = event
+                    result = self.on_event(event)
+                if asyncio.iscoroutine(result):
+                    result = await result
+                results.append(result)
+                if event_status == 'completed':
+                    self.responded_events.pop(event_id)
+                else:
+                    self.responded_events[event_id] = result
+            for i, result in enumerate(results):
+                if result not in ('optIn', 'optOut') and events[i]['response_required'] == 'always':
+                    logger.error("Your on_event or on_update_event handler must return 'optIn' or 'optOut'; "
+                                 f"you supplied {result}. Please fix your on_event handler.")
+                    results[i] = 'optOut'
+        except Exception as err:
+            logger.error("Your on_event handler encountered an error. Will Opt Out of the event. "
+                         f"The error was {err.__class__.__name__}: {str(err)}")
+            results = ['optOut'] * len(events)
+
+        event_responses = [{'response_code': 200,
+                            'response_description': 'OK',
+                            'opt_type': results[i],
+                            'request_id': message['request_id'],
+                            'modification_number': 1,
+                            'event_id': events[i]['event_descriptor']['event_id']}
+                           for i, event in enumerate(events)
+                           if event['response_required'] == 'always'
+                           and not utils.determine_event_status(event['active_period']) == 'completed']
+
+        if len(event_responses) > 0:
+            response = {'response_code': 200,
+                        'response_description': 'OK',
+                        'request_id': message['request_id']}
+            message = self._create_message('oadrCreatedEvent',
+                                           response=response,
+                                           event_responses=event_responses,
+                                           ven_id=self.ven_id)
+            service = 'EiEvent'
+            response_type, response_payload = await self._perform_request(service, message)
+            logger.info(response_type, response_payload)
+        else:
+            logger.info("Not sending any event responses, because a response was not required/allowed by the VTN.")
+
+    async def _event_cleanup(self):
+        """
+        Periodic task that will clean up completed events in our memory.
+        """
+        print("Checking for stale events")
+        for event in list(self.received_events):
+            if utils.determine_event_status(self.received_events[event]['active_period']) == 'completed':
+                logger.debug(f"Removing event {event} because it is completed.")
+                self.received_events.pop(event)
 
     async def _poll(self):
-        print("Now polling")
+        logger.debug("Now polling for new messages")
         response_type, response_payload = await self.poll()
         if response_type is None:
             return
 
         if response_type == 'oadrResponse':
-            print("No events or reports available")
+            logger.debug("No events or reports available")
             return
 
         if response_type == 'oadrRequestReregistration':
+            logger.info("The VTN required us to re-register. Calling the registration procedure.")
             await self.create_party_registration()
 
         if response_type == 'oadrDistributeEvent':
-            await self._on_event(response_payload)
+            if len(response_payload['events']) > 0:
+                await self._on_event(response_payload)
 
         elif response_type == 'oadrUpdateReport':
             await self._on_report(response_payload)
 
+        elif response_type == 'oadrCreateReport':
+            if 'report_requests' in response_payload:
+                for report_request in response_payload['report_requests']:
+                    await self.create_report(report_request)
+
+        elif response_type == 'oadrRegisterReport':
+            if 'reports' in response_payload and len(response_payload['reports']) > 0:
+                for report in response_payload['reports']:
+                    await self.register_report(report)
+
         else:
-            print(f"No handler implemented for message type {response_type}, ignoring.")
+            logger.warning(f"No handler implemented for incoming message "
+                           f"of type {response_type}, ignoring.")
 
         # Immediately poll again, because there might be more messages
         await self._poll()
+
+    async def _ensure_client_session(self):
+        if not self.client_session:
+            headers = {'content-type': 'application/xml'}
+            if self.cert_path:
+                ssl_context = ssl.create_default_context(cafile=self.ca_file,
+                                                         purpose=ssl.Purpose.CLIENT_AUTH)
+                ssl_context.load_cert_chain(self.cert_path, self.key_path, self.passphrase)
+                ssl_context.check_hostname = False
+                connector = aiohttp.TCPConnector(ssl=ssl_context)
+                self.client_session = aiohttp.ClientSession(connector=connector, headers=headers)
+            else:
+                self.client_session = aiohttp.ClientSession(headers=headers)

openleadr/enums.py

@@ -18,6 +18,8 @@
 A collection of useful enumerations that you can use to construct or
 interpret OpenADR messages. Can also be useful during testing.
 """
+from openleadr.objects import Measurement, PowerAttributes
+
 
 class Enum(type):
     def __getitem__(self, item):
@@ -25,12 +27,14 @@ class Enum(type):
 
     @property
     def members(self):
-        return sorted([item for item in list(set(dir(self)) - set(dir(Enum))) if not item.startswith("_")])
+        return sorted([item for item in list(set(dir(self)) - set(dir(Enum)))
+                       if not item.startswith("_")])
 
     @property
     def values(self):
         return [self[item] for item in self.members]
 
+
 class EVENT_STATUS(metaclass=Enum):
     NONE = "none"
     FAR = "far"
@@ -39,6 +43,7 @@ class EVENT_STATUS(metaclass=Enum):
     COMPLETED = "completed"
     CANCELLED = "cancelled"
 
+
 class SIGNAL_TYPE(metaclass=Enum):
     DELTA = "delta"
     LEVEL = "level"
@@ -49,9 +54,10 @@ class SIGNAL_TYPE(metaclass=Enum):
     SETPOINT = "setpoint"
     X_LOAD_CONTROL_CAPACITY = "x-loadControlCapacity"
     X_LOAD_CONTROL_LEVEL_OFFSET = "x-loadControlLevelOffset"
-    X_LOAD_CONTROL_PERCENT_OFFSET = "x-loadControlPorcentOffset"
+    X_LOAD_CONTROL_PERCENT_OFFSET = "x-loadControlPercentOffset"
     X_LOAD_CONTROL_SETPOINT = "x-loadControlSetpoint"
 
+
 class SIGNAL_NAME(metaclass=Enum):
     SIMPLE = "SIMPLE"
     simple = "simple"
@@ -65,6 +71,7 @@ class SIGNAL_NAME(metaclass=Enum):
     LOAD_DISPATCH = "LOAD_DISPATCH"
     LOAD_CONTROL = "LOAD_CONTROL"
 
+
 class SI_SCALE_CODE(metaclass=Enum):
     p = "p"
     n = "n"
@@ -78,10 +85,12 @@ class SI_SCALE_CODE(metaclass=Enum):
     T = "T"
     none = "none"
 
+
 class OPT(metaclass=Enum):
     OPT_IN = "optIn"
     OPT_OUT = "optOut"
 
+
 class OPT_REASON(metaclass=Enum):
     ECONOMIC = "economic"
     EMERGENCY = "emergency"
@@ -92,6 +101,7 @@ class OPT_REASON(metaclass=Enum):
     PARTICIPATING = "participating"
     X_SCHEDULE = "x-schedule"
 
+
 class READING_TYPE(metaclass=Enum):
     DIRECT_READ = "Direct Read"
     NET = "Net"
@@ -107,6 +117,7 @@ class READING_TYPE(metaclass=Enum):
     X_RMS = "x-RMS"
     X_NOT_APPLICABLE = "x-notApplicable"
 
+
 class REPORT_TYPE(metaclass=Enum):
     READING = "reading"
     USAGE = "usage"
@@ -133,6 +144,32 @@ class REPORT_TYPE(metaclass=Enum):
     PERCENT_DEMAND = "percentDemand"
     X_RESOURCE_STATUS = "x-resourceStatus"
 
+
+class SIGNAL_TARGET_MRID(metaclass=Enum):
+    THERMOSTAT = "Thermostat"
+    STRIP_HEATER = "Strip_Heater"
+    BASEBOARD_HEATER = "Baseboard_Heater"
+    WATER_HEATER = "Water_Heater"
+    POOL_PUMP = "Pool_Pump"
+    SAUNA = "Sauna"
+    HOT_TUB = "Hot_tub"
+    SMART_APPLIANCE = "Smart_Appliance"
+    IRRIGATION_PUMP = "Irrigation_Pump"
+    MANAGED_COMMERCIAL_AND_INDUSTRIAL_LOADS = "Managed_Commercial_and_Industrial_Loads"
+    SIMPLE_RESIDENTIAL_ON_OFF_LOADS = "Simple_Residential_On_Off_Loads"
+    EXTERIOR_LIGHTING = "Exterior_Lighting"
+    INTERIOR_LIGHTING = "Interior_Lighting"
+    ELECTRIC_VEHICLE = "Electric_Vehicle"
+    GENERATION_SYSTEMS = "Generation_Systems"
+    LOAD_CONTROL_SWITCH = "Load_Control_Switch"
+    SMART_INVERTER = "Smart_Inverter"
+    EVSE = "EVSE"
+    RESU = "RESU"
+    ENERGY_MANAGEMENT_SYSTEM = "Energy_Management_System"
+    SMART_ENERGY_MODULE = "Smart_Energy_Module"
+    STORAGE = "Storage"
+
+
 class REPORT_NAME(metaclass=Enum):
     METADATA_HISTORY_USAGE = "METADATA_HISTORY_USAGE"
     HISTORY_USAGE = "HISTORY_USAGE"
@@ -143,19 +180,238 @@ class REPORT_NAME(metaclass=Enum):
     METADATA_TELEMETRY_STATUS = "METADATA_TELEMETRY_STATUS"
     TELEMETRY_STATUS = "TELEMETRY_STATUS"
 
+
 class STATUS_CODES(metaclass=Enum):
-    OUT_OF_SEQUENCE  = 450
-    NOT_ALLOWED      = 451
-    INVALID_ID       = 452
-    NOT_RECOGNIZED   = 453
-    INVALID_DATA     = 454
+    OUT_OF_SEQUENCE = 450
+    NOT_ALLOWED = 451
+    INVALID_ID = 452
+    NOT_RECOGNIZED = 453
+    INVALID_DATA = 454
     COMPLIANCE_ERROR = 459
     SIGNAL_NOT_SUPPORTED = 460
     REPORT_NOT_SUPPORTED = 461
     TARGET_MISMATCH = 462
     NOT_REGISTERED_OR_AUTHORIZED = 463
-    DEPLOYMENT_ERROR_OTHER = 469
+    DEPLOYMENT_ERROR_OR_OTHER_ERROR = 469
+
 
 class SECURITY_LEVEL:
-    STANDARD: 'STANDARD'
-    HIGH: 'HIGH'
+    STANDARD = 'STANDARD'
+    HIGH = 'HIGH'
+
+
+_CURRENCIES = ("AFN", "ALL", "AMD", "ANG", "AOA", "ARS", "AUD", "AWG", "AZN", "BAM",
+               "BBD", "BDT", "BGN", "BHD", "BIF", "BMD", "BND", "BOB", "BOV", "BRL",
+               "BSD", "BTN", "BWP", "BYR", "BZD", "CAD", "CDF", "CHE", "CHF", "CHW",
+               "CLF", "CLP", "CNY", "COP", "COU", "CRC", "CUC", "CUP", "CVE", "CZK",
+               "DJF", "DKK", "DOP", "DZD", "EEK", "EGP", "ERN", "ETB", "EUR", "FJD",
+               "FKP", "GBP", "GEL", "GHS", "GIP", "GMD", "GNF", "GTQ", "GWP", "GYD",
+               "HKD", "HNL", "HRK", "HTG", "HUF", "IDR", "ILS", "INR", "IQD", "IRR",
+               "ISK", "JMD", "JOD", "JPY", "KES", "KGS", "KHR", "KMF", "KPW", "KRW",
+               "KWD", "KYD", "KZT", "LAK", "LBP", "LKR", "LRD", "LSL", "LTL", "LVL",
+               "LYD", "MAD", "MAD", "MDL", "MGA", "MKD", "MMK", "MNT", "MOP", "MRO",
+               "MUR", "MVR", "MWK", "MXN", "MXV", "MYR", "MZN", "NAD", "NGN", "NIO",
+               "NOK", "NPR", "NZD", "OMR", "PAB", "PEN", "PGK", "PHP", "PKR", "PLN",
+               "PYG", "QAR", "RON", "RSD", "RUB", "RWF", "SAR", "SBD", "SCR", "SDG",
+               "SEK", "SGD", "SHP", "SLL", "SOS", "SRD", "STD", "SVC", "SYP", "SZL",
+               "THB", "TJS", "TMT", "TND", "TOP", "TRY", "TTD", "TWD", "TZS", "UAH",
+               "UGX", "USD", "USN", "USS", "UYI", "UYU", "UZS", "VEF", "VND", "VUV",
+               "WST", "XAF", "XAG", "XAU", "XBA", "XBB", "XBC", "XBD", "XCD", "XDR",
+               "XFU", "XOF", "XPD", "XPF", "XPF", "XPF", "XPT", "XTS", "XXX", "YER",
+               "ZAR", "ZMK", "ZWL")
+
+_ACCEPTABLE_UNITS = {'currency': _CURRENCIES,
+                     'currencyPerKW': _CURRENCIES,
+                     'currencyPerKWh': _CURRENCIES,
+                     'currencyPerThm': _CURRENCIES,
+                     'current': ('A',),
+                     'energyApparent': ('VAh',),
+                     'energyReactive': ('VARh',),
+                     'energyReal': ('Wh',),
+                     'frequency': ('Hz',),
+                     'powerApparent': ('VA',),
+                     'powerReactive': ('VAR',),
+                     'powerReal': ('W',),
+                     'pulseCount': ('count',),
+                     'temperature': ('celsius', 'fahrenheit'),
+                     'Therm': ('thm',),
+                     'voltage': ('V',)}
+
+_MEASUREMENT_DESCRIPTIONS = {'currency': 'currency',
+                             'currencyPerKW': 'currencyPerKW',
+                             'currencyPerKWh': 'currencyPerKWh',
+                             'currencyPerThm': 'currency',
+                             'current': 'Current',
+                             'energyApparent': 'ApparentEnergy',
+                             'energyReactive': 'ReactiveEnergy',
+                             'energyReal': 'RealEnergy',
+                             'frequency': 'Frequency',
+                             'powerApparent': 'ApparentPower',
+                             'powerReactive': 'ReactivePower',
+                             'powerReal': 'RealPower',
+                             'pulseCount': 'pulse count',
+                             'temperature': 'temperature',
+                             'Therm': 'Therm',
+                             'voltage': 'Voltage'}
+
+_MEASUREMENT_NAMESPACES = {'currency': 'oadr',
+                           'currencyPerWK': 'oadr',
+                           'currencyPerKWh': 'oadr',
+                           'currencyPerThm': 'oadr',
+                           'current': 'oadr',
+                           'energyApparent': 'power',
+                           'energyReactive': 'power',
+                           'energyReal': 'power',
+                           'frequency': 'oadr',
+                           'powerApparent': 'power',
+                           'powerReactive': 'power',
+                           'powerReal': 'power',
+                           'pulseCount': 'oadr',
+                           'temperature': 'oadr',
+                           'Therm': 'oadr',
+                           'voltage': 'power',
+                           'customUnit': 'oadr'}
+
+
+class MEASUREMENTS(metaclass=Enum):
+    VOLTAGE = Measurement(name='voltage',
+                          description=_MEASUREMENT_DESCRIPTIONS['voltage'],
+                          unit=_ACCEPTABLE_UNITS['voltage'][0],
+                          acceptable_units=_ACCEPTABLE_UNITS['voltage'],
+                          scale='none')
+    CURRENT = Measurement(name='current',
+                          description=_MEASUREMENT_DESCRIPTIONS['current'],
+                          unit=_ACCEPTABLE_UNITS['current'][0],
+                          acceptable_units=_ACCEPTABLE_UNITS['current'],
+                          scale='none')
+    ENERGY_REAL = Measurement(name='energyReal',
+                              description=_MEASUREMENT_DESCRIPTIONS['energyReal'],
+                              unit=_ACCEPTABLE_UNITS['energyReal'][0],
+                              acceptable_units=_ACCEPTABLE_UNITS['energyReal'],
+                              scale='none')
+    REAL_ENERGY = Measurement(name='energyReal',
+                              description=_MEASUREMENT_DESCRIPTIONS['energyReal'],
+                              unit=_ACCEPTABLE_UNITS['energyReal'][0],
+                              acceptable_units=_ACCEPTABLE_UNITS['energyReal'],
+                              scale='none')
+    ACTIVE_ENERGY = Measurement(name='energyReal',
+                                description=_MEASUREMENT_DESCRIPTIONS['energyReal'],
+                                unit=_ACCEPTABLE_UNITS['energyReal'][0],
+                                acceptable_units=_ACCEPTABLE_UNITS['energyReal'],
+                                scale='none')
+    ENERGY_REACTIVE = Measurement(name='energyReactive',
+                                  description=_MEASUREMENT_DESCRIPTIONS['energyReactive'],
+                                  unit=_ACCEPTABLE_UNITS['energyReactive'][0],
+                                  acceptable_units=_ACCEPTABLE_UNITS['energyReactive'],
+                                  scale='none')
+    REACTIVE_ENERGY = Measurement(name='energyReactive',
+                                  description=_MEASUREMENT_DESCRIPTIONS['energyReactive'],
+                                  unit=_ACCEPTABLE_UNITS['energyReactive'][0],
+                                  acceptable_units=_ACCEPTABLE_UNITS['energyReactive'],
+                                  scale='none')
+    ENERGY_APPARENT = Measurement(name='energyApparent',
+                                  description=_MEASUREMENT_DESCRIPTIONS['energyApparent'],
+                                  unit=_ACCEPTABLE_UNITS['energyApparent'][0],
+                                  acceptable_units=_ACCEPTABLE_UNITS['energyApparent'],
+                                  scale='none')
+    APPARENT_ENERGY = Measurement(name='energyApparent',
+                                  description=_MEASUREMENT_DESCRIPTIONS['energyApparent'],
+                                  unit=_ACCEPTABLE_UNITS['energyApparent'][0],
+                                  acceptable_units=_ACCEPTABLE_UNITS['energyApparent'],
+                                  scale='none')
+    ACTIVE_POWER = Measurement(name='powerReal',
+                               description=_MEASUREMENT_DESCRIPTIONS['powerReal'],
+                               unit=_ACCEPTABLE_UNITS['powerReal'][0],
+                               acceptable_units=_ACCEPTABLE_UNITS['powerReal'],
+                               scale='none',
+                               power_attributes=PowerAttributes(hertz=50,
+                                                                voltage=230,
+                                                                ac=True))
+    REAL_POWER = Measurement(name='powerReal',
+                             description=_MEASUREMENT_DESCRIPTIONS['powerReal'],
+                             unit=_ACCEPTABLE_UNITS['powerReal'][0],
+                             acceptable_units=_ACCEPTABLE_UNITS['powerReal'],
+                             scale='none',
+                             power_attributes=PowerAttributes(hertz=50,
+                                                              voltage=230,
+                                                              ac=True))
+    POWER_REAL = Measurement(name='powerReal',
+                             description=_MEASUREMENT_DESCRIPTIONS['powerReal'],
+                             unit=_ACCEPTABLE_UNITS['powerReal'][0],
+                             acceptable_units=_ACCEPTABLE_UNITS['powerReal'],
+                             scale='none',
+                             power_attributes=PowerAttributes(hertz=50,
+                                                              voltage=230,
+                                                              ac=True))
+    REACTIVE_POWER = Measurement(name='powerReactive',
+                                 description=_MEASUREMENT_DESCRIPTIONS['powerReactive'],
+                                 unit=_ACCEPTABLE_UNITS['powerReactive'][0],
+                                 acceptable_units=_ACCEPTABLE_UNITS['powerReactive'],
+                                 scale='none',
+                                 power_attributes=PowerAttributes(hertz=50,
+                                                                  voltage=230,
+                                                                  ac=True))
+    POWER_REACTIVE = Measurement(name='powerReactive',
+                                 description=_MEASUREMENT_DESCRIPTIONS['powerReactive'],
+                                 unit=_ACCEPTABLE_UNITS['powerReactive'][0],
+                                 acceptable_units=_ACCEPTABLE_UNITS['powerReactive'],
+                                 scale='none',
+                                 power_attributes=PowerAttributes(hertz=50,
+                                                                  voltage=230,
+                                                                  ac=True))
+    APPARENT_POWER = Measurement(name='powerApparent',
+                                 description=_MEASUREMENT_DESCRIPTIONS['powerApparent'],
+                                 unit=_ACCEPTABLE_UNITS['powerApparent'][0],
+                                 acceptable_units=_ACCEPTABLE_UNITS['powerApparent'],
+                                 scale='none',
+                                 power_attributes=PowerAttributes(hertz=50,
+                                                                  voltage=230,
+                                                                  ac=True))
+    POWER_APPARENT = Measurement(name='powerApparent',
+                                 description=_MEASUREMENT_DESCRIPTIONS['powerApparent'],
+                                 unit=_ACCEPTABLE_UNITS['powerApparent'][0],
+                                 acceptable_units=_ACCEPTABLE_UNITS['powerApparent'],
+                                 scale='none',
+                                 power_attributes=PowerAttributes(hertz=50,
+                                                                  voltage=230,
+                                                                  ac=True))
+    FREQUENCY = Measurement(name='frequency',
+                            description=_MEASUREMENT_DESCRIPTIONS['frequency'],
+                            unit=_ACCEPTABLE_UNITS['frequency'][0],
+                            acceptable_units=_ACCEPTABLE_UNITS['frequency'],
+                            scale='none')
+    PULSE_COUNT = Measurement(name='pulseCount',
+                              description=_MEASUREMENT_DESCRIPTIONS['pulseCount'],
+                              unit=_ACCEPTABLE_UNITS['pulseCount'][0],
+                              acceptable_units=_ACCEPTABLE_UNITS['pulseCount'],
+                              pulse_factor=1000)
+    TEMPERATURE = Measurement(name='temperature',
+                              description=_MEASUREMENT_DESCRIPTIONS['temperature'],
+                              unit=_ACCEPTABLE_UNITS['temperature'][0],
+                              acceptable_units=_ACCEPTABLE_UNITS['temperature'],
+                              scale='none')
+    THERM = Measurement(name='Therm',
+                        description=_MEASUREMENT_DESCRIPTIONS['Therm'],
+                        unit=_ACCEPTABLE_UNITS['Therm'][0],
+                        acceptable_units=_ACCEPTABLE_UNITS['Therm'],
+                        scale='none')
+    CURRENCY = Measurement(name='currency',
+                           description=_MEASUREMENT_DESCRIPTIONS['currency'],
+                           unit=_CURRENCIES[0],
+                           acceptable_units=_CURRENCIES,
+                           scale='none')
+    CURRENCY_PER_KW = Measurement(name='currencyPerKW',
+                                  description=_MEASUREMENT_DESCRIPTIONS['currencyPerKW'],
+                                  unit=_CURRENCIES[0],
+                                  acceptable_units=_CURRENCIES,
+                                  scale='none')
+    CURRENCY_PER_KWH = Measurement(name='currencyPerKWh',
+                                   description=_MEASUREMENT_DESCRIPTIONS['currencyPerKWh'],
+                                   unit=_CURRENCIES[0],
+                                   acceptable_units=_CURRENCIES,
+                                   scale='none')
+    CURRENCY_PER_THM = Measurement(name='currencyPerThm',
+                                   description=_MEASUREMENT_DESCRIPTIONS['currencyPerThm'],
+                                   unit=_CURRENCIES[0],
+                                   acceptable_units=_CURRENCIES,
+                                   scale='none')

openleadr/errors.py

@@ -1,27 +1,93 @@
-# SPDX-License-Identifier: Apache-2.0
+from openleadr.enums import STATUS_CODES
 
-# Copyright 2020 Contributors to OpenLEADR
 
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+class ProtocolError(Exception):
+    pass
 
-#     http://www.apache.org/licenses/LICENSE-2.0
 
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+class FingerprintMismatch(Exception):
+    pass
 
-from .enums import STATUS_CODES
 
-class OpenADRError(Exception):
-    def __init__(self, status, description):
-        assert status in self.status_codes.values, f"Invalid status code {status} while raising OpenADRError"
+class HTTPError(Exception):
+    def __init__(self, status=500, description=None):
         super().__init__()
-        self.status = status
-        self.description = description
+        self.response_code = status
+        self.response_description = description
 
-    def __str__(self):
-        return f'Error {self.status} {self.status_codes[self.status]}: {self.description}'
+
+class OutOfSequenceError(ProtocolError):
+    def __init__(self, description='OUT OF SEQUENCE'):
+        super().__init__()
+        self.response_code = STATUS_CODES.OUT_OF_SEQUENCE
+        self.response_description = description
+
+
+class NotAllowedError(ProtocolError):
+    def __init__(self, description='NOT ALLOWED'):
+        super().__init__()
+        self.response_code = STATUS_CODES.NOT_ALLOWED
+        self.response_description = description
+
+
+class InvalidIdError(ProtocolError):
+    def __init__(self, description='INVALID ID'):
+        super().__init__()
+        self.response_code = STATUS_CODES.INVALID_ID
+        self.response_description = description
+
+
+class NotRecognizedError(ProtocolError):
+    def __init__(self, description='NOT RECOGNIZED'):
+        super().__init__()
+        self.response_code = STATUS_CODES.NOT_RECOGNIZED
+        self.response_description = description
+
+
+class InvalidDataError(ProtocolError):
+    def __init__(self, description='INVALID DATA'):
+        super().__init__()
+        self.response_code = STATUS_CODES.INVALID_DATA
+        self.response_description = description
+
+
+class ComplianceError(ProtocolError):
+    def __init__(self, description='COMPLIANCE ERROR'):
+        super().__init__()
+        self.response_code = STATUS_CODES.COMPLIANCE_ERROR
+        self.response_description = description
+
+
+class SignalNotSupportedError(ProtocolError):
+    def __init__(self, description='SIGNAL NOT SUPPORTED'):
+        super().__init__()
+        self.response_code = STATUS_CODES.SIGNAL_NOT_SUPPORTED
+        self.response_description = description
+
+
+class ReportNotSupportedError(ProtocolError):
+    def __init__(self, description='REPORT NOT SUPPORTED'):
+        super().__init__()
+        self.response_code = STATUS_CODES.REPORT_NOT_SUPPORTED
+        self.response_description = description
+
+
+class TargetMismatchError(ProtocolError):
+    def __init__(self, description='TARGET MISMATCH'):
+        super().__init__()
+        self.response_code = STATUS_CODES.TARGET_MISMATCH
+        self.response_description = description
+
+
+class NotRegisteredOrAuthorizedError(ProtocolError):
+    def __init__(self, description='NOT REGISTERED OR AUTHORIZED'):
+        super().__init__()
+        self.response_code = STATUS_CODES.NOT_REGISTERED_OR_AUTHORIZED
+        self.response_description = description
+
+
+class DeploymentError(ProtocolError):
+    def __init__(self, description='DEPLOYMENT ERROR OR OTHER ERROR'):
+        super().__init__()
+        self.response_code = STATUS_CODES.DEPLOYMENT_ERROR_OR_OTHER_ERROR
+        self.response_description = description

openleadr/fingerprint.py

@@ -0,0 +1,13 @@
+from argparse import ArgumentParser
+from openleadr.utils import certificate_fingerprint
+
+
+def show_fingerprint():
+    parser = ArgumentParser()
+    parser.add_argument('certificate', type=str)
+    args = parser.parse_args()
+
+    if 'certificate' in args:
+        with open(args.certificate) as file:
+            cert_str = file.read()
+            print(certificate_fingerprint(cert_str))

openleadr/messaging.py

@@ -20,47 +20,56 @@ from jinja2 import Environment, PackageLoader
 from signxml import XMLSigner, XMLVerifier, methods
 from uuid import uuid4
 from lxml.etree import Element
+from asyncio import iscoroutine
+from openleadr import errors
+from datetime import datetime, timezone, timedelta
+import os
 
-from .utils import *
+from openleadr import utils
 from .preflight import preflight_message
 
-from dataclasses import is_dataclass, asdict
+import logging
+logger = logging.getLogger('openleadr')
+
 SIGNER = XMLSigner(method=methods.detached,
-                   c14n_algorithm="http://www.w3.org/2001/10/xml-exc-c14n#")
+                   c14n_algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315")
+SIGNER.namespaces['oadr'] = "http://openadr.org/oadr-2.0b/2012/07"
 VERIFIER = XMLVerifier()
 
-def parse_message(data, fingerprint=None, fingerprint_lookup=None):
+XML_SCHEMA_LOCATION = os.path.join(os.path.dirname(__file__), 'schema', 'oadr_20b.xsd')
+
+with open(XML_SCHEMA_LOCATION) as file:
+    XML_SCHEMA = etree.XMLSchema(etree.parse(file))
+XML_PARSER = etree.XMLParser(schema=XML_SCHEMA)
+
+
+def parse_message(data):
     """
     Parse a message and distill its usable parts. Returns a message type and payload.
+    :param data str: The XML string that is received
+
+    Returns a message type (str) and a message payload (dict)
     """
     message_dict = xmltodict.parse(data, process_namespaces=True, namespaces=NAMESPACES)
     message_type, message_payload = message_dict['oadrPayload']['oadrSignedObject'].popitem()
-    if 'ven_id' in message_payload:
-        _validate_and_authenticate_message(data, message_dict, fingerprint, fingerprint_lookup)
-    return message_type, normalize_dict(message_payload)
+    message_payload = utils.normalize_dict(message_payload)
+    return message_type, message_payload
+
 
 def create_message(message_type, cert=None, key=None, passphrase=None, **message_payload):
     """
     Create and optionally sign an OpenADR message. Returns an XML string.
     """
-    # If we supply the payload as dataclasses, convert them to dicts
-    for k, v in message_payload.items():
-        if isinstance(v, list):
-            for i, item in enumerate(v):
-                if is_dataclass(item):
-                    v[i] = asdict(item)
-        elif is_dataclass(v):
-            message_payload[k] = asdict(v)
-
-    preflight_message(message_type, message_payload)
-    signed_object = flatten_xml(TEMPLATES.get_template(f'{message_type}.xml').render(**message_payload))
+    message_payload = preflight_message(message_type, message_payload)
+    template = TEMPLATES.get_template(f'{message_type}.xml')
+    signed_object = utils.flatten_xml(template.render(**message_payload))
     envelope = TEMPLATES.get_template('oadrPayload.xml')
     if cert and key:
         tree = etree.fromstring(signed_object)
         signature_tree = SIGNER.sign(tree,
                                      key=key,
                                      cert=cert,
-                                     passphrase=ensure_bytes(passphrase),
+                                     passphrase=utils.ensure_bytes(passphrase),
                                      reference_uri="#oadrSignedObject",
                                      signature_properties=_create_replay_protect())
         signature = etree.tostring(signature_tree).decode('utf-8')
@@ -72,24 +81,81 @@ def create_message(message_type, cert=None, key=None, passphrase=None, **message
                           signed_object=signed_object)
     return msg
 
-def _validate_and_authenticate_message(data, message_dict, fingerprint=None, fingerprint_lookup=None):
-    if not fingerprint and not fingerprint_lookup:
-        return
-    tree = etree.fromstring(ensure_bytes(data))
-    cert = extract_pem_cert(tree)
-    ven_id = tree.find('.//{http://docs.oasis-open.org/ns/energyinterop/201110}venID').text
-    cert_fingerprint = certificate_fingerprint(cert)
-    if not fingerprint:
-        fingerprint = fingerprint_lookup(ven_id)
-
-    if fingerprint != certificate_fingerprint(cert):
-        raise ValueError("The fingerprint does not match")
-    VERIFIER.verify(tree, x509_cert=ensure_bytes(cert), expect_references=2)
-    _verify_replay_protect(message_dict)
+
+def validate_xml_schema(content):
+    """
+    Validates the XML tree against the schema. Return the XML tree.
+    """
+    if isinstance(content, str):
+        content = content.encode('utf-8')
+    tree = etree.fromstring(content, XML_PARSER)
+    return tree
+
+
+def validate_xml_signature(xml_tree, cert_fingerprint=None):
+    """
+    Validate the XMLDSIG signature and the ReplayProtect element.
+    """
+    cert = utils.extract_pem_cert(xml_tree)
+    if cert_fingerprint:
+        fingerprint = utils.certificate_fingerprint(cert)
+        if fingerprint != cert_fingerprint:
+            raise errors.FingerprintMismatch("The certificate fingerprint was incorrect. "
+                                             f"Expected: {cert_fingerprint};"
+                                             f"Received: {fingerprint}")
+    VERIFIER.verify(xml_tree, x509_cert=utils.ensure_bytes(cert), expect_references=2)
+    _verify_replay_protect(xml_tree)
+
+
+async def authenticate_message(request, message_tree, message_payload, fingerprint_lookup):
+    if request.secure and 'ven_id' in message_payload:
+        connection_fingerprint = utils.get_cert_fingerprint_from_request(request)
+        if connection_fingerprint is None:
+            msg = ("Your request must use a client side SSL certificate, of which the "
+                   "fingerprint must match the fingerprint that you have given to this VTN.")
+            raise errors.NotRegisteredOrAuthorizedError(msg)
+
+        try:
+            ven_id = message_payload.get('ven_id')
+            expected_fingerprint = fingerprint_lookup(ven_id)
+            if iscoroutine(expected_fingerprint):
+                expected_fingerprint = await expected_fingerprint
+        except ValueError:
+            msg = (f"Your venID {ven_id} is not known to this VTN. Make sure you use the venID "
+                   "that you receive from this VTN during the registration step")
+            raise errors.NotRegisteredOrAuthorizedError(msg)
+
+        if expected_fingerprint is None:
+            msg = ("This VTN server does not know what your certificate fingerprint is. Please "
+                   "deliver your fingerprint to the VTN (outside of OpenADR). You used the "
+                   "following fingerprint to make this request:")
+            raise errors.NotRegisteredOrAuthorizedError(msg)
+
+        if connection_fingerprint != expected_fingerprint:
+            msg = (f"The fingerprint of your HTTPS certificate '{connection_fingerprint}' "
+                   f"does not match the expected fingerprint '{expected_fingerprint}'")
+            raise errors.NotRegisteredOrAuthorizedError(msg)
+
+        message_cert = utils.extract_pem_cert(message_tree)
+        message_fingerprint = utils.certificate_fingerprint(message_cert)
+        if message_fingerprint != expected_fingerprint:
+            msg = (f"The fingerprint of the certificate used to sign the message "
+                   f"{message_fingerprint} did not match the fingerprint that this "
+                   f"VTN has for you {expected_fingerprint}. Make sure you use the correct "
+                   "certificate to sign your messages.")
+            raise errors.NotRegisteredOrAuthorizedError(msg)
+
+        try:
+            validate_xml_signature(message_tree)
+        except ValueError:
+            msg = ("The message signature did not match the message contents. Please make sure "
+                   "you are using the correct XMLDSig algorithm and C14n canonicalization.")
+            raise errors.NotRegisteredOrAuthorizedError(msg)
+
 
 def _create_replay_protect():
     dt_element = Element("{http://openadr.org/oadr-2.0b/2012/07/xmldsig-properties}timestamp")
-    dt_element.text = datetimeformat(datetime.now(timezone.utc))
+    dt_element.text = utils.datetimeformat(datetime.now(timezone.utc))
 
     nonce_element = Element("{http://openadr.org/oadr-2.0b/2012/07/xmldsig-properties}nonce")
     nonce_element.text = uuid4().hex
@@ -101,25 +167,30 @@ def _create_replay_protect():
     el.append(nonce_element)
     return el
 
-def _verify_replay_protect(message_dict):
+
+def _verify_replay_protect(xml_tree):
     try:
-        ts = message_dict['oadrPayload']['Signature']['Object']['SignatureProperties']['SignatureProperty']['ReplayProtect']['timestamp']
-        nonce = message_dict['oadrPayload']['Signature']['Object']['SignatureProperties']['SignatureProperty']['ReplayProtect']['nonce']
-    except KeyError:
-        raise ValueError("Missing ReplayProtect")
+        ns = "{http://openadr.org/oadr-2.0b/2012/07/xmldsig-properties}"
+        timestamp = utils.parse_datetime(xml_tree.findtext(f".//{ns}timestamp"))
+        nonce = xml_tree.findtext(f".//{ns}nonce")
+    except Exception:
+        raise ValueError("Missing or malformed ReplayProtect element in the message signature.")
     else:
-        timestamp = datetime.strptime(ts, "%Y-%m-%dT%H:%M:%S.%f%z")
+        if nonce is None:
+            raise ValueError("Missing 'nonce' element in ReplayProtect in incoming message.")
         if timestamp < datetime.now(timezone.utc) - REPLAY_PROTECT_MAX_TIME_DELTA:
-            raise ValueError("Message is too old")
+            raise ValueError("The message was signed too long ago.")
         elif (timestamp, nonce) in NONCE_CACHE:
-            raise ValueError("This combination of timestamp and nonce was already used")
+            raise ValueError("This combination of timestamp and nonce was already used.")
     _update_nonce_cache(timestamp, nonce)
 
+
 def _update_nonce_cache(timestamp, nonce):
+    NONCE_CACHE.add((timestamp, nonce))
     for timestamp, nonce in list(NONCE_CACHE):
         if timestamp < datetime.now(timezone.utc) - REPLAY_PROTECT_MAX_TIME_DELTA:
             NONCE_CACHE.remove((timestamp, nonce))
-    NONCE_CACHE.add((timestamp, nonce))
+
 
 # Replay protect settings
 REPLAY_PROTECT_MAX_TIME_DELTA = timedelta(seconds=5)
@@ -127,9 +198,9 @@ NONCE_CACHE = set()
 
 # Settings for jinja2
 TEMPLATES = Environment(loader=PackageLoader('openleadr', 'templates'))
-TEMPLATES.filters['datetimeformat'] = datetimeformat
-TEMPLATES.filters['timedeltaformat'] = timedeltaformat
-TEMPLATES.filters['booleanformat'] = booleanformat
+TEMPLATES.filters['datetimeformat'] = utils.datetimeformat
+TEMPLATES.filters['timedeltaformat'] = utils.timedeltaformat
+TEMPLATES.filters['booleanformat'] = utils.booleanformat
 TEMPLATES.trim_blocks = True
 TEMPLATES.lstrip_blocks = True
 

openleadr/objects.py

@@ -14,48 +14,60 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from dataclasses import dataclass
-from typing import List
+from dataclasses import dataclass, field, asdict, is_dataclass
+from typing import List, Dict
 from datetime import datetime, timezone, timedelta
+from openleadr import utils
+from openleadr import enums
+
 
 @dataclass
 class AggregatedPNode:
     node: str
 
+
 @dataclass
 class EndDeviceAsset:
     mrid: str
 
+
 @dataclass
 class MeterAsset:
     mrid: str
 
+
 @dataclass
 class PNode:
     node: str
 
+
 @dataclass
 class FeatureCollection:
     id: str
     location: dict
 
+
 @dataclass
 class ServiceArea:
     feature_collection: FeatureCollection
 
+
 @dataclass
 class ServiceDeliveryPoint:
     node: str
 
+
 @dataclass
 class ServiceLocation:
     node: str
 
+
 @dataclass
 class TransportInterface:
     point_of_receipt: str
     point_of_delivery: str
 
+
 @dataclass
 class Target:
     aggregated_p_node: AggregatedPNode = None
@@ -72,9 +84,15 @@ class Target:
     ven_id: str = None
     party_id: str = None
 
+    def __repr__(self):
+        targets = {key: value for key, value in asdict(self).items() if value is not None}
+        targets_str = ", ".join(f"{key}={value}" for key, value in targets.items())
+        return f"Target('{targets_str}')"
+
+
 @dataclass
 class EventDescriptor:
-    event_id: int
+    event_id: str
     modification_number: int
     market_context: str
     event_status: str
@@ -86,7 +104,6 @@ class EventDescriptor:
     vtn_comment: str = None
 
     def __post_init__(self):
-        print("Calling Post Init")
         if self.modification_date_time is None:
             self.modification_date_time = datetime.now(timezone.utc)
         if self.created_date_time is None:
@@ -94,14 +111,16 @@ class EventDescriptor:
         if self.modification_number is None:
             self.modification_number = 0
 
+
 @dataclass
 class ActivePeriod:
     dtstart: datetime
     duration: timedelta
     tolerance: dict = None
-    notification: dict = None
-    ramp_up: dict = None
-    recovery: dict = None
+    notification_period: dict = None
+    ramp_up_period: dict = None
+    recovery_period: dict = None
+
 
 @dataclass
 class Interval:
@@ -110,24 +129,177 @@ class Interval:
     signal_payload: float
     uid: int = None
 
+
+@dataclass
+class SamplingRate:
+    min_period: timedelta = None
+    max_period: timedelta = None
+    on_change: bool = False
+
+
+@dataclass
+class PowerAttributes:
+    hertz: int = 50
+    voltage: int = 230
+    ac: bool = True
+
+
+@dataclass
+class Measurement:
+    name: str
+    description: str
+    unit: str
+    acceptable_units: List[str] = field(repr=False, default_factory=list)
+    scale: str = None
+    power_attributes: PowerAttributes = None
+    pulse_factor: int = None
+    ns: str = 'power'
+
+    def __post_init__(self):
+        if self.name not in enums._MEASUREMENT_NAMESPACES:
+            self.name = 'customUnit'
+        self.ns = enums._MEASUREMENT_NAMESPACES[self.name]
+
+
 @dataclass
 class EventSignal:
     intervals: List[Interval]
     signal_name: str
     signal_type: str
     signal_id: str
-    current_value: float
+    current_value: float = None
     targets: List[Target] = None
+    targets_by_type: Dict = None
+    measurement: Measurement = None
+
+    def __post_init__(self):
+        if self.targets is None and self.targets_by_type is None:
+            return
+        elif self.targets_by_type is None:
+            list_of_targets = [asdict(target) if is_dataclass(target) else target for target in self.targets]
+            targets_by_type = utils.group_targets_by_type(list_of_targets)
+            if len(targets_by_type) > 1:
+                raise ValueError("In OpenADR, the EventSignal target may only be of type endDeviceAsset. "
+                                 f"You provided types: {', '.join(targets_by_type)}")
+        elif self.targets is None:
+            self.targets = [Target(**target) for target in utils.ungroup_targets_by_type(self.targets_by_type)]
+        elif self.targets is not None and self.targets_by_type is not None:
+            list_of_targets = [asdict(target) if is_dataclass(target) else target for target in self.targets]
+            if utils.group_targets_by_type(list_of_targets) != self.targets_by_type:
+                raise ValueError("You assigned both 'targets' and 'targets_by_type' in your event, "
+                                 "but the two were not consistent with each other. "
+                                 f"You supplied 'targets' = {self.targets} and "
+                                 f"'targets_by_type' = {self.targets_by_type}")
+
 
 @dataclass
 class Event:
     event_descriptor: EventDescriptor
-    active_period: ActivePeriod
-    event_signals: EventSignal
-    targets: List[Target]
+    event_signals: List[EventSignal]
+    targets: List[Target] = None
+    targets_by_type: Dict = None
+    active_period: ActivePeriod = None
+    response_required: str = 'always'
+
+    def __post_init__(self):
+        if self.active_period is None:
+            dtstart = min([i['dtstart']
+                           if isinstance(i, dict) else i.dtstart
+                           for s in self.event_signals for i in s.intervals])
+            duration = max([i['dtstart'] + i['duration']
+                            if isinstance(i, dict) else i.dtstart + i.duration
+                            for s in self.event_signals for i in s.intervals]) - dtstart
+            self.active_period = ActivePeriod(dtstart=dtstart,
+                                              duration=duration)
+        if self.targets is None and self.targets_by_type is None:
+            raise ValueError("You must supply either 'targets' or 'targets_by_type'.")
+        elif self.targets_by_type is None:
+            list_of_targets = [asdict(target) if is_dataclass(target) else target for target in self.targets]
+            self.targets_by_type = utils.group_targets_by_type(list_of_targets)
+        elif self.targets is None:
+            self.targets = [Target(**target) for target in utils.ungroup_targets_by_type(self.targets_by_type)]
+        elif self.targets is not None and self.targets_by_type is not None:
+            list_of_targets = [asdict(target) if is_dataclass(target) else target for target in self.targets]
+            if utils.group_targets_by_type(list_of_targets) != self.targets_by_type:
+                raise ValueError("You assigned both 'targets' and 'targets_by_type' in your event, "
+                                 "but the two were not consistent with each other. "
+                                 f"You supplied 'targets' = {self.targets} and "
+                                 f"'targets_by_type' = {self.targets_by_type}")
+        # Set the event status
+        self.event_descriptor.event_status = utils.determine_event_status(self.active_period)
+
 
 @dataclass
 class Response:
     response_code: int
     response_description: str
     request_id: str
+
+
+@dataclass
+class ReportDescription:
+    r_id: str                           # Identifies a specific datapoint in a report
+    market_context: str
+    reading_type: str
+    report_subject: Target
+    report_data_source: Target
+    report_type: str
+    sampling_rate: SamplingRate
+    measurement: Measurement = None
+
+
+@dataclass
+class ReportPayload:
+    r_id: str
+    value: float
+    confidence: int = None
+    accuracy: int = None
+
+
+@dataclass
+class ReportInterval:
+    dtstart: datetime
+    report_payload: ReportPayload
+    duration: timedelta = None
+
+
+@dataclass
+class Report:
+    report_specifier_id: str            # This is what the VEN calls this report
+    report_name: str                    # Usually one of the default ones (enums.REPORT_NAME)
+    report_request_id: str = None       # Usually empty
+    report_descriptions: List[ReportDescription] = None
+    created_date_time: datetime = None
+
+    dtstart: datetime = None                # For delivering values
+    duration: timedelta = None              # For delivering values
+    intervals: List[ReportInterval] = None  # For delivering values
+    data_collection_mode: str = 'incremental'
+
+    def __post_init__(self):
+        if self.created_date_time is None:
+            self.created_date_time = datetime.now(timezone.utc)
+        if self.report_descriptions is None:
+            self.report_descriptions = []
+
+
+@dataclass
+class SpecifierPayload:
+    r_id: str
+    reading_type: str
+    measurement: Measurement = None
+
+
+@dataclass
+class ReportSpecifier:
+    report_specifier_id: str    # This is what the VEN called this report
+    granularity: timedelta
+    specifier_payloads: List[SpecifierPayload]
+    report_interval: Interval = None
+    report_back_duration: timedelta = None
+
+
+@dataclass
+class ReportRequest:
+    report_request_id: str
+    report_specifier: ReportSpecifier

openleadr/preflight.py

@@ -15,7 +15,11 @@
 # limitations under the License.
 
 from datetime import datetime, timedelta, timezone
-import warnings
+from dataclasses import asdict, is_dataclass
+from openleadr import enums, utils
+import logging
+logger = logging.getLogger('openleadr')
+
 
 def preflight_message(message_type, message_payload):
     """
@@ -27,8 +31,39 @@ def preflight_message(message_type, message_payload):
     :param message_payload dict: The contents of the message
     """
     if f'_preflight_{message_type}' in globals():
+        message_payload = message_payload.copy()
+        for key, value in message_payload.items():
+            if isinstance(value, list):
+                message_payload[key] = [asdict(item) if is_dataclass(item) else item
+                                        for item in value]
+            else:
+                message_payload[key] = asdict(value) if is_dataclass(value) else value
         globals()[f'_preflight_{message_type}'](message_payload)
-    return message_type, message_payload
+    return message_payload
+
+
+def _preflight_oadrRegisterReport(message_payload):
+    for report in message_payload['reports']:
+        # Check that the report name is preceded by METADATA_ when registering reports
+        if report['report_name'] in enums.REPORT_NAME.values \
+                and not report['report_name'].startswith("METADATA"):
+            report['report_name'] = 'METADATA_' + report['report_name']
+
+        # Check that the measurement name and description match according to the schema
+        for report_description in report['report_descriptions']:
+            if 'measurement' in report_description and report_description['measurement'] is not None:
+                utils.validate_report_measurement_dict(report_description['measurement'])
+
+        # Add the correct namespace to the measurement
+        for report_description in report['report_descriptions']:
+            if 'measurement' in report_description and report_description['measurement'] is not None:
+                if report_description['measurement']['name'] in enums._MEASUREMENT_NAMESPACES:
+                    measurement_name = report_description['measurement']['name']
+                    measurement_ns = enums._MEASUREMENT_NAMESPACES[measurement_name]
+                    report_description['measurement']['ns'] = measurement_ns
+                else:
+                    raise ValueError("The Measurement Name is unknown")
+
 
 def _preflight_oadrDistributeEvent(message_payload):
     if 'parse_duration' not in globals():
@@ -38,15 +73,19 @@ def _preflight_oadrDistributeEvent(message_payload):
         active_period_duration = event['active_period']['duration']
         signal_durations = []
         for signal in event['event_signals']:
-            signal_durations.append(sum([parse_duration(i['duration']) for i in signal['intervals']], timedelta(seconds=0)))
+            signal_durations.append(sum([parse_duration(i['duration'])
+                                         for i in signal['intervals']], timedelta(seconds=0)))
 
-        if not all([d==active_period_duration for d in signal_durations]):
-            if not all([d==signal_durations[0] for d in signal_durations]):
-                raise ValueError("The different EventSignals have different total durations. Please correct this.")
+        if not all([d == active_period_duration for d in signal_durations]):
+            if not all([d == signal_durations[0] for d in signal_durations]):
+                raise ValueError("The different EventSignals have different total durations. "
+                                 "Please correct this.")
             else:
-                warnings.warn(f"The active_period duration for event {event['event_descriptor']['event_id']} ({active_period_duration})"
-                              f" was different from the sum of the interval's durations ({signal_durations[0]})."
-                              f" The active_period duration has been adjusted to ({signal_durations[0]}).")
+                logger.warning(f"The active_period duration for event "
+                               f"{event['event_descriptor']['event_id']} ({active_period_duration})"
+                               f" differs from the sum of the interval's durations "
+                               f"({signal_durations[0]}). The active_period duration has been "
+                               f"adjusted to ({signal_durations[0]}).")
                 event['active_period']['duration'] = signal_durations[0]
 
     # Check that payload values with signal name SIMPLE are constricted (rule 9)
@@ -55,27 +94,56 @@ def _preflight_oadrDistributeEvent(message_payload):
             if event_signal['signal_name'] == "SIMPLE":
                 for interval in event_signal['intervals']:
                     if interval['signal_payload'] not in (0, 1, 2, 3):
-                        raise ValueError("Payload Values used with Signal Name SIMPLE must be one of"
-                                         "0, 1, 2 or 3")
+                        raise ValueError("Payload Values used with Signal Name SIMPLE "
+                                         "must be one of 0, 1, 2 or 3")
 
     # Check that the current_value is 0 for SIMPLE events that are not yet active (rule 14)
     for event in message_payload['events']:
         for event_signal in event['event_signals']:
             if 'current_value' in event_signal and event_signal['current_value'] != 0:
-                if event_signal['signal_name'] == "SIMPLE" and event['event_descriptor']['event_status'] != "ACTIVE":
-                    warnings.warn("The current_value for a SIMPLE event that is not yet active must be 0. This will be corrected.")
+                if event_signal['signal_name'] == "SIMPLE" \
+                        and event['event_descriptor']['event_status'] != "ACTIVE":
+                    logger.warning("The current_value for a SIMPLE event "
+                                   "that is not yet active must be 0. "
+                                   "This will be corrected.")
                     event_signal['current_value'] = 0
 
+    # Add the correct namespace to the measurement
+    for event in message_payload['events']:
+        for event_signal in event['event_signals']:
+            if 'measurement' in event_signal and event_signal['measurement'] is not None:
+                if event_signal['measurement']['name'] in enums._MEASUREMENT_NAMESPACES:
+                    measurement_name = event_signal['measurement']['name']
+                    measurement_ns = enums._MEASUREMENT_NAMESPACES[measurement_name]
+                    event_signal['measurement']['ns'] = measurement_ns
+                else:
+                    raise ValueError("The Measurement Name is unknown")
+
     # Check that there is a valid oadrResponseRequired value for each Event
     for event in message_payload['events']:
         if 'response_required' not in event:
             event['response_required'] = 'always'
         elif event['response_required'] not in ('never', 'always'):
-            warnings.warn(f"The response_required property in an Event should be 'never' or 'always', not {event['response_required']}. Changing to 'always'.")
+            logger.warning(f"The response_required property in an Event "
+                           f"should be 'never' or 'always', not "
+                           f"{event['response_required']}. Changing to 'always'.")
             event['response_required'] = 'always'
 
     # Check that there is a valid oadrResponseRequired value for each Event
     for event in message_payload['events']:
-        if 'created_date_time' not in event['event_descriptor'] or not event['event_descriptor']['created_date_time']:
-            print("ADDING CREATED DATE TIME")
+        if 'created_date_time' not in event['event_descriptor'] \
+                or not event['event_descriptor']['created_date_time']:
+            logger.warning("Your event descriptor did not contain a created_date_time. "
+                           "This will be automatically added.")
             event['event_descriptor']['created_date_time'] = datetime.now(timezone.utc)
+
+    # Check that the target designations are correct and consistent
+    for event in message_payload['events']:
+        if 'targets' in event and 'targets_by_type' in event:
+            if utils.group_targets_by_type(event['targets']) != event['targets_by_type']:
+                raise ValueError("You assigned both 'targets' and 'targets_by_type' in your event, "
+                                 "but the two were not consistent with each other. "
+                                 f"You supplied 'targets' = {event['targets']} and "
+                                 f"'targets_by_type' = {event['targets_by_type']}")
+        elif 'targets_by_type' in event and 'targets' not in event:
+            event['targets'] = utils.ungroup_targets_by_type(event['targets_by_type'])

openleadr/schema/LICENSES.txt

@@ -0,0 +1,128 @@
+This file contains the license information for some of the included XSD files,
+which are owned by others.
+
+oadr_xmldsig11.xsd:
+    Licensed under the W3C license. The W3C license agreement is copied below.
+
+oadr_siscale_20b.xsd
+    This file is © OASIS Open 2012 and contains a copyright notice. The text of
+    that notice is copied below.
+
+oadr_ISO_ISO3AlphaCurrencyCode_20100407.xsd
+    Licensed under the UN_CEFACT license. The license agreement is listed inside
+    that file.
+
+Other XSDs included in this directory do not contain explicit license
+information, and are included here in good faith and believing that we are
+allowed allowed to include them here. If you feel like we are missing an
+attribution, we will do everything we can to correct this problem. Please reach
+out to us at team@openleadr.org or file an issue on our GitHub page at
+https://github.com/openleadr/openleadr-python/issues. Thank you.
+
+
+================================= W3C Licence =================================
+
+This work is being provided by the copyright holders under the following
+license. License
+
+By obtaining and/or copying this work, you (the licensee) agree that you have
+read, understood, and will comply with the following terms and conditions.
+
+Permission to copy, modify, and distribute this work, with or without
+modification, for any purpose and without fee or royalty is hereby granted,
+provided that you include the following on ALL copies of the work or portions
+thereof, including modifications:
+
+    The full text of this NOTICE in a location viewable to users of the
+    redistributed or derivative work. Any pre-existing intellectual property
+    disclaimers, notices, or terms and conditions. If none exist, the W3C
+    Software and Document Short Notice should be included. Notice of any changes
+    or modifications, through a copyright statement on the new code or document
+    such as "This software or document includes material copied from or derived
+    from [title and URI of the W3C document]. Copyright © [YEAR] W3C® (MIT,
+    ERCIM, Keio, Beihang)."
+
+Disclaimers
+
+THIS WORK IS PROVIDED "AS IS," AND COPYRIGHT HOLDERS MAKE NO REPRESENTATIONS OR
+WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO, WARRANTIES OF
+MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE
+SOFTWARE OR DOCUMENT WILL NOT INFRINGE ANY THIRD PARTY PATENTS, COPYRIGHTS,
+TRADEMARKS OR OTHER RIGHTS.
+
+COPYRIGHT HOLDERS WILL NOT BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL OR
+CONSEQUENTIAL DAMAGES ARISING OUT OF ANY USE OF THE SOFTWARE OR DOCUMENT.
+
+The name and trademarks of copyright holders may NOT be used in advertising or
+publicity pertaining to the work without specific, written prior permission.
+Title to copyright in this work will at all times remain with copyright holders.
+
+-------------------------------------------------------------------------------
+
+
+========================= OASIS Open Copyright Notice =========================
+
+Copyright © OASIS Open 2012. All Rights Reserved.
+
+All capitalized terms in the following text have the meanings assigned to them
+in the OASIS Intellectual Property Rights Policy (the "OASIS IPR Policy"). The
+full Policy may be found at the OASIS website.
+
+This document and translations of it may be copied and furnished to others, and
+derivative works that comment on or otherwise explain it or assist in its
+implementation may be prepared, copied, published, and distributed, in whole or
+in part, without restriction of any kind, provided that the above copyright
+notice and this section are included on all such copies and derivative works.
+However, this document itself may not be modified in any way, including by
+removing the copyright notice or references to OASIS, except as needed for the
+purpose of developing any document or deliverable produced by an OASIS Technical
+Committee (in which case the rules applicable to copyrights, as set forth in the
+OASIS IPR Policy, must be followed) or as required to translate it into
+languages other than English.
+
+The limited permissions granted above are perpetual and will not be revoked by
+OASIS or its successors or assigns.
+
+This document and the information contained herein is provided on an "AS IS"
+basis and OASIS DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT
+LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE
+ANY OWNERSHIP RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR
+A PARTICULAR PURPOSE.
+
+OASIS requests that any OASIS Party or any other party that believes it has
+patent claims that would necessarily be infringed by implementations of this
+OASIS Committee Specification or OASIS Standard, to notify OASIS TC
+Administrator and provide an indication of its willingness to grant patent
+licenses to such patent claims in a manner consistent with the IPR Mode of the
+OASIS Technical Committee that produced this specification.
+
+OASIS invites any party to contact the OASIS TC Administrator if it is aware of
+a claim of ownership of any patent claims that would necessarily be infringed by
+implementations of this specification by a patent holder that is not willing to
+provide a license to such patent claims in a manner consistent with the IPR Mode
+of the OASIS Technical Committee that produced this specification. OASIS may
+include such claims on its website, but disclaims any obligation to do so.
+
+OASIS takes no position regarding the validity or scope of any intellectual
+property or other rights that might be claimed to pertain to the implementation
+or use of the technology described in this document or the extent to which any
+license under such rights might or might not be available; neither does it
+represent that it has made any effort to identify any such rights. Information
+on OASIS' procedures with respect to rights in any document or deliverable
+produced by an OASIS Technical Committee can be found on the OASIS website.
+Copies of claims of rights made available for publication and any assurances of
+licenses to be made available, or the result of an attempt made to obtain a
+general license or permission for the use of such proprietary rights by
+implementers or users of this OASIS Committee Specification or OASIS Standard,
+can be obtained from the OASIS TC Administrator. OASIS makes no representation
+that any information or list of intellectual property rights will at any time be
+complete, or that any claims in such list are, in fact, Essential Claims.
+
+The name "OASIS" is a trademark of OASIS, the owner and developer of this
+specification, and should be used only to refer to the organization and its
+official outputs. OASIS welcomes reference to, and implementation and use of,
+specifications, while reserving the right to enforce its marks against
+misleading uses. Please see http://www.oasis-open.org/who/trademark.php for
+above guidance.
+
+-------------------------------------------------------------------------------

openleadr/server.py

@@ -15,45 +15,97 @@
 # limitations under the License.
 
 from aiohttp import web
-from openleadr.service import EventService, PollService, RegistrationService, ReportService, OptService, VTNService
-from openleadr.messaging import create_message, parse_message
-from openleadr.utils import certificate_fingerprint
+from openleadr.service import EventService, PollService, RegistrationService, ReportService, \
+                              VTNService
+from openleadr.messaging import create_message
+from openleadr import objects
+from openleadr import utils
 from functools import partial
+from datetime import datetime, timedelta, timezone
+import asyncio
+import inspect
+import logging
+import ssl
+import re
+import sys
+logger = logging.getLogger('openleadr')
+
 
 class OpenADRServer:
-    _MAP = {'on_created_event': EventService,
-            'on_request_event': EventService,
+    _MAP = {'on_created_event': 'event_service',
+            'on_request_event': 'event_service',
 
-            'on_register_report': ReportService,
-            'on_create_report': ReportService,
-            'on_created_report': ReportService,
-            'on_request_report': ReportService,
-            'on_update_report': ReportService,
+            'on_register_report': 'report_service',
+            'on_create_report': 'report_service',
+            'on_created_report': 'report_service',
+            'on_request_report': 'report_service',
+            'on_update_report': 'report_service',
 
-            'on_poll': PollService,
+            'on_poll': 'poll_service',
 
-            'on_query_registration': RegistrationService,
-            'on_create_party_registration': RegistrationService,
-            'on_cancel_party_registration': RegistrationService}
+            'on_query_registration': 'registration_service',
+            'on_create_party_registration': 'registration_service',
+            'on_cancel_party_registration': 'registration_service'}
 
-    def __init__(self, vtn_id, cert=None, key=None, passphrase=None, fingerprint_lookup=None):
+    def __init__(self, vtn_id, cert=None, key=None, passphrase=None, fingerprint_lookup=None,
+                 show_fingerprint=True, http_port=8080, http_host='127.0.0.1', http_cert=None,
+                 http_key=None, http_key_passphrase=None, http_path_prefix='/OpenADR2/Simple/2.0b',
+                 requested_poll_freq=timedelta(seconds=10), http_ca_file=None):
         """
         Create a new OpenADR VTN (Server).
 
-        :param vtn_id string: An identifier string for this VTN. This is how you identify yourself to the VENs that talk to you.
-        :param cert string: Path to the PEM-formatted certificate file that is used to sign outgoing messages
-        :param key string: Path to the PEM-formatted private key file that is used to sign outgoing messages
-        :param passphrase string: The passphrase used to decrypt the private key file
-        :param fingerprint_lookup callable: A callable that receives a ven_id and should return the registered fingerprint for that VEN.
-                                            You should receive these fingerprints outside of OpenADR and configure them manually.
+        :param str vtn_id: An identifier string for this VTN. This is how you identify yourself
+                              to the VENs that talk to you.
+        :param str cert: Path to the PEM-formatted certificate file that is used to sign outgoing
+                            messages
+        :param str key: Path to the PEM-formatted private key file that is used to sign outgoing
+                           messages
+        :param str passphrase: The passphrase used to decrypt the private key file
+        :param callable fingerprint_lookup: A callable that receives a ven_id and should return the
+                                            registered fingerprint for that VEN. You should receive
+                                            these fingerprints outside of OpenADR and configure them
+                                            manually.
+        :param bool show_fingerprint: Whether to print the fingerprint to your stdout on startup.
+                                         Defaults to True.
+        :param int http_port: The port that the web server is exposed on (default: 8080)
+        :param str http_host: The host or IP address to bind the server to (default: 127.0.0.1).
+        :param str http_cert: The path to the PEM certificate for securing HTTP traffic.
+        :param str http_key: The path to the PEM private key for securing HTTP traffic.
+        :param str http_ca_file: The path to the CA-file that client certificates are checked against.
+        :param str http_key_passphrase: The passphrase for the HTTP private key.
         """
+        # Set up the message queues
+
         self.app = web.Application()
-        self.services = {'event_service': EventService(vtn_id),
-                         'report_service': ReportService(vtn_id),
-                         'poll_service': PollService(vtn_id),
-                         'opt_service': OptService(vtn_id),
-                         'registration_service': RegistrationService(vtn_id)}
-        self.app.add_routes([web.post(f"/OpenADR2/Simple/2.0b/{s.__service_name__}", s.handler) for s in self.services.values()])
+        self.services = {}
+        self.services['event_service'] = EventService(vtn_id)
+        self.services['report_service'] = ReportService(vtn_id)
+        self.services['poll_service'] = PollService(vtn_id)
+        self.services['registration_service'] = RegistrationService(vtn_id, poll_freq=requested_poll_freq)
+
+        # Register the other services with the poll service
+        self.services['poll_service'].event_service = self.services['event_service']
+        self.services['poll_service'].report_service = self.services['report_service']
+
+        # Set up the HTTP handlers for the services
+        if http_path_prefix[-1] == "/":
+            http_path_prefix = http_path_prefix[:-1]
+        self.app.add_routes([web.post(f"{http_path_prefix}/{s.__service_name__}", s.handler)
+                             for s in self.services.values()])
+
+        # Configure the web server
+        self.http_port = http_port
+        self.http_host = http_host
+        self.http_path_prefix = http_path_prefix
+
+        # Create SSL context for running the server
+        if http_cert and http_key:
+            self.ssl_context = ssl.create_default_context(cafile=http_ca_file,
+                                                          purpose=ssl.Purpose.CLIENT_AUTH)
+            self.ssl_context.verify_mode = ssl.CERT_REQUIRED
+            self.ssl_context.load_cert_chain(http_cert, http_key, http_key_passphrase)
+        else:
+            self.ssl_context = None
 
         # Configure message signing
         if cert and key:
@@ -61,35 +113,198 @@ class OpenADRServer:
                 cert = file.read()
             with open(key, "rb") as file:
                 key = file.read()
-            print("*" * 80)
-            print("Your VTN Certificate Fingerprint is", certificate_fingerprint(cert))
-            print("Please deliver this fingerprint to the VTN you are connecting to.")
-            print("You do not need to keep this a secret.")
-            print("*" * 80)
+            if show_fingerprint:
+                print("")
+                print("*" * 80)
+                print("Your VTN Certificate Fingerprint is "
+                      f"{utils.certificate_fingerprint(cert)}".center(80))
+                print("Please deliver this fingerprint to the VENs that connect to you.".center(80))
+                print("You do not need to keep this a secret.".center(80))
+                print("*" * 80)
+                print("")
+        VTNService._create_message = partial(create_message, cert=cert, key=key,
+                                             passphrase=passphrase)
+        VTNService.fingerprint_lookup = staticmethod(fingerprint_lookup)
+        self.__setattr__ = self.add_handler
 
-        VTNService._create_message = partial(create_message, cert=cert, key=key, passphrase=passphrase)
-        VTNService._parse_message = partial(parse_message, fingerprint_lookup=fingerprint_lookup)
+    async def run(self):
+        """
+        Starts the server in an already-running asyncio loop.
+        """
+        self.app_runner = web.AppRunner(self.app)
+        await self.app_runner.setup()
+        site = web.TCPSite(self.app_runner,
+                           port=self.http_port,
+                           host=self.http_host,
+                           ssl_context=self.ssl_context)
+        await site.start()
+        protocol = 'https' if self.ssl_context else 'http'
+        print("")
+        print("*" * 80)
+        print("Your VTN Server is now running at ".center(80))
+        print(f"{protocol}://{self.http_host}:{self.http_port}{self.http_path_prefix}".center(80))
+        print("*" * 80)
+        print("")
 
-        self.__setattr__ = self.add_handler
+    async def run_async(self):
+        await self.run()
+
+    async def stop(self):
+        if sys.version_info.minor >= 8:
+            delayed_call_tasks = [task for task in asyncio.all_tasks()
+                                  if task.get_name().startswith('DelayedCall')]
+            for task in delayed_call_tasks:
+                task.cancel()
+        await self.app_runner.cleanup()
+
+    def add_event(self, ven_id, signal_name, signal_type, intervals, callback=None, event_id=None,
+                  targets=None, targets_by_type=None, target=None, response_required='always',
+                  market_context="oadr://unknown.context", notification_period=None,
+                  ramp_up_period=None, recovery_period=None, signal_target_mrid=None):
+        """
+        Convenience method to add an event with a single signal.
 
-    def run(self):
+        :param str ven_id: The ven_id to whom this event must be delivered.
+        :param str signal_name: The OpenADR name of the signal; one of openleadr.objects.SIGNAL_NAME
+        :param str signal_type: The OpenADR type of the signal; one of openleadr.objects.SIGNAL_TYPE
+        :param str intervals: A list of intervals with a dtstart, duration and payload member.
+        :param str callback: A callback function for when your event has been accepted (optIn) or refused (optOut).
+        :param list targets: A list of Targets that this Event applies to.
+        :param target: A single target for this event.
+        :param dict targets_by_type: A dict of targets, grouped by type.
+        :param str market_context: A URI for the DR program that this event belongs to.
+        :param timedelta notification_period: The Notification period for the Event's Active Period.
+        :param timedelta ramp_up_period: The Ramp Up period for the Event's Active Period.
+        :param timedelta recovery_period: The Recovery period for the Event's Active Period.
+
+        If you don't provide a target using any of the three arguments, the target will be set to the given ven_id.
+        """
+        if self.services['event_service'].polling_method == 'external':
+            logger.error("You cannot use the add_event method after you assign your own on_poll "
+                         "handler. If you use your own on_poll handler, you are responsible for "
+                         "delivering events from that handler. If you want to use OpenLEADRs "
+                         "message queuing system, you should not assign an on_poll handler. "
+                         "Your Event will NOT be added.")
+            return
+        if not re.match(r"^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?", market_context):
+            raise ValueError("The Market Context must be a valid URI.")
+        event_id = event_id or utils.generate_id()
+
+        if response_required not in ('always', 'never'):
+            raise ValueError("'response_required' should be either 'always' or 'never'; "
+                             f"you provided '{response_required}'.")
+
+        # Figure out the target for this Event
+        if target is None and targets is None and targets_by_type is None:
+            targets = [{'ven_id': ven_id}]
+        elif target is not None:
+            targets = [target]
+        elif targets_by_type is not None:
+            targets = utils.ungroup_targets_by_type(targets_by_type)
+        if not isinstance(targets, list):
+            targets = [targets]
+
+        event_descriptor = objects.EventDescriptor(event_id=event_id,
+                                                   modification_number=0,
+                                                   market_context=market_context,
+                                                   event_status="far",
+                                                   created_date_time=datetime.now(timezone.utc))
+        event_signal = objects.EventSignal(intervals=intervals,
+                                           signal_name=signal_name,
+                                           signal_type=signal_type,
+                                           signal_id=utils.generate_id())
+
+        # Make sure the intervals carry timezone-aware timestamps
+        for interval in intervals:
+            if utils.getmember(interval, 'dtstart').tzinfo is None:
+                utils.setmember(interval, 'dtstart',
+                                utils.getmember(interval, 'dtstart').astimezone(timezone.utc))
+                logger.warning("You supplied a naive datetime object to your interval's dtstart. "
+                               "This will be interpreted as a timestamp in your local timezone "
+                               "and then converted to UTC before sending. Please supply timezone-"
+                               "aware timestamps like datetime.datetime.new(timezone.utc) or "
+                               "datetime.datetime(..., tzinfo=datetime.timezone.utc)")
+        active_period = utils.get_active_period_from_intervals(intervals, False)
+        active_period.ramp_up_period = ramp_up_period
+        active_period.notification_period = notification_period
+        active_period.recovery_period = recovery_period
+        event = objects.Event(active_period=active_period,
+                              event_descriptor=event_descriptor,
+                              event_signals=[event_signal],
+                              targets=targets,
+                              response_required=response_required)
+        self.add_raw_event(ven_id=ven_id, event=event, callback=callback)
+        return event_id
+
+    def add_raw_event(self, ven_id, event, callback=None):
         """
-        Starts the asyncio-loop and runs the server in it. This function is
-        blocking. For other ways to run the server in a more flexible context,
-        please refer to the `aiohttp documentation
-        <https://docs.aiohttp.org/en/stable/web_advanced.html#aiohttp-web-app-runners>`_.
+        Add a new event to the queue for a specific VEN.
+        :param str ven_id: The ven_id to which this event should be distributed.
+        :param dict event: The event (as a dict or as a objects.Event instance)
+                           that contains the event details.
+        :param callable callback: A callback that will receive the opt status for this event.
+                                  This callback receives ven_id, event_id, opt_type as its arguments.
         """
-        web.run_app(self.app)
+        if utils.getmember(event, 'response_required') == 'always':
+            if callback is None:
+                logger.warning("You did not provide a 'callback', which means you won't know if the "
+                               "VEN will opt in or opt out of your event. You should consider adding "
+                               "a callback for this.")
+            elif not asyncio.isfuture(callback):
+                args = inspect.signature(callback).parameters
+                if not all(['ven_id' in args, 'event_id' in args, 'opt_type' in args]):
+                    raise ValueError("The 'callback' must have at least the following parameters: "
+                                     "'ven_id' (str), 'event_id' (str), 'opt_type' (str). Please fix "
+                                     "your 'callback' handler.")
+
+        event_id = utils.getmember(utils.getmember(event, 'event_descriptor'), 'event_id')
+        # Create the event queue if it does not exist yet
+        if ven_id not in self.events:
+            self.events[ven_id] = []
+
+        # Add event to the queue
+        self.events[ven_id].append(event)
+        self.events_updated[ven_id] = True
+
+        # Add the callback for the response to this event
+        if callback is not None:
+            self.event_callbacks[event_id] = (event, callback)
+        return event_id
 
     def add_handler(self, name, func):
         """
         Add a handler to the OpenADRServer.
 
-        :param name string: The name for this handler. Should be one of: on_created_event, on_request_event, on_register_report, on_create_report, on_created_report, on_request_report, on_update_report, on_poll, on_query_registration, on_create_party_registration, on_cancel_party_registration.
-        :param func coroutine: A coroutine that handles this event. It receives the message, and should return the contents of a response.
+        :param str name: The name for this handler. Should be one of: on_created_event,
+                            on_request_event, on_register_report, on_create_report,
+                            on_created_report, on_request_report, on_update_report, on_poll,
+                            on_query_registration, on_create_party_registration,
+                            on_cancel_party_registration.
+        :param callable func: A function or coroutine that handles this type of occurrence.
+                              It receives the message, and should return the contents of a response.
         """
-        print("Called add_handler", name, func)
+        logger.debug(f"Adding handler: {name} {func}")
         if name in self._MAP:
-            setattr(self._MAP[name], name, staticmethod(func))
+            setattr(self.services[self._MAP[name]], name, func)
+            if name == 'on_poll':
+                self.services['poll_service'].polling_method = 'external'
+                self.services['event_service'].polling_method = 'external'
         else:
-            raise NameError(f"Unknown handler {name}. Correct handler names are: {self._MAP.keys()}")
+            raise NameError(f"""Unknown handler '{name}'. """
+                            f"""Correct handler names are: '{"', '".join(self._MAP.keys())}'.""")
+
+    @property
+    def registered_reports(self):
+        return self.services['report_service'].registered_reports
+
+    @property
+    def events(self):
+        return self.services['event_service'].events
+
+    @property
+    def events_updated(self):
+        return self.services['poll_service'].events_updated
+
+    @property
+    def event_callbacks(self):
+        return self.services['event_service'].event_callbacks

openleadr/service/__init__.py

@@ -14,28 +14,12 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-def handler(message_type):
-    """
-    Decorator to mark a method as the handler for a specific message type.
-    """
-    def _actual_decorator(decorated_function):
-        decorated_function.__message_type__ = message_type
-        return decorated_function
-    return _actual_decorator
+# flake8: noqa
 
-def service(service_name):
-    """
-    Decorator to mark a class as an OpenADR Service for a specific endpoint.
-    """
-    def _actual_decorator(decorated_function):
-        decorated_function.__service_name__ = service_name
-        return decorated_function
-    return _actual_decorator
-
-# The classes below all register to the api
+from .decorators import handler, service
 from .vtn_service import VTNService
 from .event_service import EventService
 from .poll_service import PollService
 from .registration_service import RegistrationService
 from .report_service import ReportService
-from .opt_service import OptService
+from .opt_service import OptService

openleadr/config.py → openleadr/service/decorators.py

@@ -14,8 +14,21 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from os.path import abspath, dirname, join
+def handler(message_type):
+    """
+    Decorator to mark a method as the handler for a specific message type.
+    """
+    def _actual_decorator(decorated_function):
+        decorated_function.__message_type__ = message_type
+        return decorated_function
+    return _actual_decorator
 
-VTN_ID = "elaadvtn"
-DATABASE = "openadr"
-TEMPLATE_DIR = join(abspath(dirname(__file__)), 'templates')
+
+def service(service_name):
+    """
+    Decorator to mark a class as an OpenADR Service for a specific endpoint.
+    """
+    def _actual_decorator(decorated_function):
+        decorated_function.__service_name__ = service_name
+        return decorated_function
+    return _actual_decorator

openleadr/service/event_service.py

@@ -15,35 +15,103 @@
 # limitations under the License.
 
 from . import service, handler, VTNService
-from datetime import datetime, timedelta, timezone
-from asyncio import iscoroutine
+import asyncio
+from openleadr import utils, errors
+import logging
+logger = logging.getLogger('openleadr')
+
 
 @service('EiEvent')
 class EventService(VTNService):
 
+    def __init__(self, vtn_id, polling_method='internal'):
+        super().__init__(vtn_id)
+        self.polling_method = polling_method
+        self.events = {}
+        self.completed_event_ids = {}   # Holds the ids of completed events
+        self.event_callbacks = {}
+        self.event_opt_types = {}
+
     @handler('oadrRequestEvent')
     async def request_event(self, payload):
         """
         The VEN requests us to send any events we have.
         """
-        result = self.on_request_event(payload['ven_id'])
-        if iscoroutine(result):
-            result = await result
-        if result is None:
-            return 'oadrDistributeEvent', {'events': []}
-        if isinstance(result, dict):
-            return 'oadrDistributeEvent', {'events': [result]}
-        if isinstance(result, list):
-            return 'oadrDistributeEvent', {'events': result}
+        ven_id = payload['ven_id']
+        if self.polling_method == 'internal':
+            if ven_id in self.events and self.events[ven_id]:
+                events = utils.order_events(self.events[ven_id])
+                for event in events:
+                    event_status = utils.getmember(utils.getmember(event, 'event_descriptor'), 'event_status')
+                    # Pop the event from the events so that this is the last time it is communicated
+                    if event_status == 'completed':
+                        self.events[ven_id].pop(self.events[ven_id].index(event))
+            else:
+                events = None
+        else:
+            result = self.on_request_event(ven_id=payload['ven_id'])
+            if asyncio.iscoroutine(result):
+                result = await result
+            if result is None:
+                events = None
+            else:
+                events = utils.order_events(result)
+
+        if events is None:
+            return 'oadrResponse', {}
         else:
-            raise TypeError("Event handler should return None, a dict or a list")
+            return 'oadrDistributeEvent', {'events': events}
+        return 'oadrResponse', result
+
+    def on_request_event(self, ven_id):
+        """
+        Placeholder for the on_request_event handler.
+        """
+        logger.warning("You should implement and register your own on_request_event handler "
+                       "that returns the next event for a VEN. This handler will receive a "
+                       "ven_id as its only argument, and should return None (if no events are "
+                       "available), a single Event, or a list of Events.")
+        return None
 
     @handler('oadrCreatedEvent')
     async def created_event(self, payload):
         """
         The VEN informs us that they created an EiEvent.
         """
-        result = self.on_created_event(payload)
-        if iscoroutine(result):
-            result = await(result)
-        return result
+        ven_id = payload['ven_id']
+        if self.polling_method == 'internal':
+            for event_response in payload['event_responses']:
+                event_id = event_response['event_id']
+                opt_type = event_response['opt_type']
+                if event_id not in [utils.getmember(utils.getmember(event, 'event_descriptor'), 'event_id')
+                                    for event in self.events[ven_id]] + self.completed_event_ids.get(ven_id, []):
+                    raise errors.InvalidIdError
+                if event_response['event_id'] in self.event_callbacks:
+                    event, callback = self.event_callbacks.pop(event_id)
+                    if isinstance(callback, asyncio.Future):
+                        if callback.done():
+                            logger.warning(f"Got a second response '{opt_type}' from ven '{ven_id}' "
+                                           f"to event '{event_id}', which we cannot use because the "
+                                           "callback future you provided was already completed during "
+                                           "the first response.")
+                        else:
+                            callback.set_result(opt_type)
+                    else:
+                        result = callback(ven_id=ven_id, event_id=event_id, opt_type=opt_type)
+                        if asyncio.iscoroutine(result):
+                            result = await result
+        else:
+            result = self.on_created_event(ven_id=ven_id, event_id=event_id, opt_type=opt_type)
+            if asyncio.iscoroutine(result):
+                result = await(result)
+        return 'oadrResponse', {}
+
+    def on_created_event(self, ven_id, event_id, opt_type):
+        """
+        Placeholder for the on_created_event handler.
+        """
+        logger.warning("You should implement and register you own on_created_event handler "
+                       "to receive the opt status for an Event that you sent to the VEN. This "
+                       "handler will receive a ven_id, event_id and opt_status. "
+                       "You don't need to return anything from this handler.")
+        return None

openleadr/service/opt_service.py

@@ -14,8 +14,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from . import service, handler, VTNService
+from . import service, VTNService
+
 
 @service('EiOpt')
 class OptService(VTNService):
-    pass
+    pass

openleadr/service/poll_service.py

@@ -14,9 +14,12 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from . import service, handler, VTNService
-from asyncio import iscoroutine
-import warnings
+from openleadr.service import service, handler, VTNService
+from openleadr import objects
+import asyncio
+from dataclasses import asdict
+import logging
+logger = logging.getLogger('openleadr')
 
 # ╔══════════════════════════════════════════════════════════════════════════╗
 # ║                             POLLING SERVICE                              ║
@@ -95,26 +98,55 @@ import warnings
 # │                                                                          │
 # └──────────────────────────────────────────────────────────────────────────┘
 
+
 @service('OadrPoll')
 class PollService(VTNService):
 
+    def __init__(self, vtn_id, polling_method='internal', event_service=None, report_service=None):
+        super().__init__(vtn_id)
+        self.polling_method = polling_method
+        self.events_updated = {}
+        self.report_requests = {}
+        self.event_service = event_service
+        self.report_service = report_service
+
     @handler('oadrPoll')
     async def poll(self, payload):
         """
-        Retrieve the messages that we have for this VEN in order.
-
-        The backend get_next_message
+        Handle the request to the oadrPoll service. This either calls a previously registered
+        `on_poll` handler, or it retrieves the next message from the internal queue.
         """
-        result = self.on_poll(ven_id=payload['ven_id'])
-        if iscoroutine(result):
+        if self.polling_method == 'external':
+            result = self.on_poll(ven_id=payload['ven_id'])
+        elif self.events_updated.get(payload['ven_id']):
+            # Send a oadrDistributeEvent whenever the events were updated
+            result = await self.event_service.request_event({'ven_id': payload['ven_id']})
+            self.events_updated[payload['ven_id']] = False
+        else:
+            return 'oadrResponse', {}
+
+        if asyncio.iscoroutine(result):
             result = await result
         if result is None:
-            return result
+            return 'oadrResponse', {}
         if isinstance(result, tuple):
             return result
         if isinstance(result, list):
             return 'oadrDistributeEvent', result
         if isinstance(result, dict) and 'event_descriptor' in result:
             return 'oadrDistributeEvent', {'events': [result]}
-        warnings.warn(f"Could not determine type of message in response to oadrPoll: {result}")
+        if isinstance(result, objects.Event):
+            return 'oadrDistributeEvent', {'events': [asdict(result)]}
+        logger.warning(f"Could not determine type of message in response to oadrPoll: {result}")
         return 'oadrResponse', result
+
+    def on_poll(self, ven_id):
+        """
+        Placeholder for the on_poll handler.
+        """
+        logger.warning("You should implement and register your own on_poll handler that "
+                       "returns the next message for the VEN. This handler receives the "
+                       "ven_id as its argument, and should return None (if no messages "
+                       "are available), an Event or list of Events, a RequestReregistration "
+                       " or RequestReport.")
+        return None

openleadr/service/registration_service.py

@@ -15,8 +15,9 @@
 # limitations under the License.
 
 from . import service, handler, VTNService
-from datetime import timedelta
 from asyncio import iscoroutine
+import logging
+logger = logging.getLogger('openleadr')
 
 # ╔══════════════════════════════════════════════════════════════════════════╗
 # ║                           REGISTRATION SERVICE                           ║
@@ -58,9 +59,14 @@ from asyncio import iscoroutine
 # │                                                                          │
 # └──────────────────────────────────────────────────────────────────────────┘
 
+
 @service('EiRegisterParty')
 class RegistrationService(VTNService):
 
+    def __init__(self, vtn_id, poll_freq):
+        super().__init__(vtn_id)
+        self.poll_freq = poll_freq
+
     @handler('oadrQueryRegistration')
     async def query_registration(self, payload):
         """
@@ -73,12 +79,10 @@ class RegistrationService(VTNService):
             return result
 
         # If you don't provide a default handler, just give out the info
-        response_payload = {'response': {'response_code': 200, 'response_description': 'OK', 'request_id': payload['request_id']},
-                            'request_id': payload['request_id'],
-                            'vtn_id': self.vtn_id,
+        response_payload = {'request_id': payload['request_id'],
                             'profiles': [{'profile_name': '2.0b',
-                                          'transports': {'transport_name': 'simpleHttp'}}],
-                            'requested_oadr_poll_freq': timedelta(seconds=5)}
+                                          'transports': [{'transport_name': 'simpleHttp'}]}],
+                            'requested_oadr_poll_freq': self.poll_freq}
         return 'oadrCreatedPartyRegistration', response_payload
 
     @handler('oadrCreatePartyRegistration')
@@ -90,12 +94,37 @@ class RegistrationService(VTNService):
         if iscoroutine(result):
             result = await result
 
-        response_type, response_payload = result
-        response_payload['response'] = {'response_code': 200,
-                                       'response_description': 'OK',
-                                       'request_id': payload['request_id']}
-        response_payload['vtn_id'] = self.vtn_id
-        return response_type, response_payload
+        if result is not False and result is not None:
+            if len(result) != 2:
+                logger.error("Your on_create_party_registration handler should return either "
+                             "'False' (if the client is rejected) or a (ven_id, registration_id) "
+                             "tuple. Will REJECT the client for now.")
+                response_payload = {}
+            else:
+                ven_id, registration_id = result
+                transports = [{'transport_name': payload['transport_name']}]
+                response_payload = {'ven_id': result[0],
+                                    'registration_id': result[1],
+                                    'profiles': [{'profile_name': payload['profile_name'],
+                                                  'transports': transports}],
+                                    'requested_oadr_poll_freq': self.poll_freq}
+        else:
+            transports = [{'transport_name': payload['transport_name']}]
+            response_payload = {'profiles': [{'profile_name': payload['profile_name'],
+                                              'transports': transports}],
+                                'requested_oadr_poll_freq': self.poll_freq}
+        return 'oadrCreatedPartyRegistration', response_payload
+
+    def on_create_party_registration(self, payload):
+        """
+        Placeholder for the on_create_party_registration handler
+        """
+        logger.warning("You should implement and register your own on_create_party_registration "
+                       "handler if you want VENs to be able to connect to you. This handler will "
+                       "receive a registration request and should return either 'False' (if the "
+                       "registration is denied) or a (ven_id, registration_id) tuple if the "
+                       "registration is accepted.")
+        return False
 
     @handler('oadrCancelPartyRegistration')
     async def cancel_party_registration(self, payload):
@@ -106,3 +135,12 @@ class RegistrationService(VTNService):
         if iscoroutine(result):
             result = await result
         return result
+
+    def on_cancel_party_registration(self, ven_id):
+        """
+        Placeholder for the on_cancel_party_registration handler.
+        """
+        logger.warning("You should implement and register your own on_cancel_party_registration "
+                       "handler that allown VENs to deregister from your VTN. This will receive a "
+                       "ven_id as its argument. You don't need to return anything.")
+        return None

openleadr/service/report_service.py

@@ -15,6 +15,11 @@
 # limitations under the License.
 
 from . import service, handler, VTNService
+from asyncio import iscoroutine
+from openleadr import objects, utils
+import logging
+import inspect
+logger = logging.getLogger('openleadr')
 
 # ╔══════════════════════════════════════════════════════════════════════════╗
 # ║                              REPORT SERVICE                              ║
@@ -54,39 +59,194 @@ from . import service, handler, VTNService
 # │                                                                          │
 # └──────────────────────────────────────────────────────────────────────────┘
 
+
 @service('EiReport')
 class ReportService(VTNService):
 
+    def __init__(self, vtn_id):
+        super().__init__(vtn_id)
+        self.report_callbacks = {}
+        self.registered_reports = {}
+
     @handler('oadrRegisterReport')
     async def register_report(self, payload):
         """
-        Register a reporting type.
+        Handle the VENs reporting capabilities.
         """
-        print("Called Registered Report")
+        report_requests = []
+        args = inspect.signature(self.on_register_report).parameters
+        if all(['ven_id' in args, 'resource_id' in args, 'measurement' in args,
+                'min_sampling_interval' in args, 'max_sampling_interval' in args,
+                'unit' in args, 'scale' in args]):
+            mode = 'compact'
+        else:
+            mode = 'full'
+
+        if payload['reports'] is None:
+            return
+
+        for report in payload['reports']:
+            if report['report_name'] == 'METADATA_TELEMETRY_STATUS':
+                if mode == 'compact':
+                    results = [self.on_register_report(ven_id=payload['ven_id'],
+                                                       resource_id=rd.get('report_data_source', {}).get('resource_id'),
+                                                       measurement='Status',
+                                                       unit=None,
+                                                       scale=None,
+                                                       min_sampling_interval=rd['sampling_rate']['min_period'],
+                                                       max_sampling_interval=rd['sampling_rate']['max_period'])
+                               for rd in report['report_descriptions']]
+                    results = await utils.gather_if_required(results)
+                elif mode == 'full':
+                    results = await utils.await_if_required(self.on_register_report(report))
+            elif report['report_name'] == 'METADATA_TELEMETRY_USAGE':
+                if mode == 'compact':
+                    results = [self.on_register_report(ven_id=payload['ven_id'],
+                                                       resource_id=rd.get('report_data_source', {}).get('resource_id'),
+                                                       measurement=rd['measurement']['description'],
+                                                       unit=rd['measurement']['unit'],
+                                                       scale=rd['measurement']['scale'],
+                                                       min_sampling_interval=rd['sampling_rate']['min_period'],
+                                                       max_sampling_interval=rd['sampling_rate']['max_period'])
+                               for rd in report['report_descriptions']]
+                    results = await utils.gather_if_required(results)
+                elif mode == 'full':
+                    results = await utils.await_if_required(self.on_register_report(report))
+            elif report['report_name'] in ('METADATA_HISTORY_USAGE', 'METADATA_HISTORY_GREENBUTTON'):
+                if payload['ven_id'] not in self.registered_reports:
+                    self.registered_reports[payload['ven_id']] = []
+                report['report_name'] = report['report_name'][9:]
+                self.registered_reports[payload['ven_id']].append(report)
+                report_requests.append(None)
+                continue
+            else:
+                logger.warning("Reports other than TELEMETRY_USAGE, TELEMETRY_STATUS, "
+                               "HISTORY_USAGE and HISTORY_GREENBUTTON are not yet supported. "
+                               f"Skipping report with name {report['report_name']}.")
+                report_requests.append(None)
+                continue
+
+            # Perform some rudimentary checks on the returned type
+            if results is not None:
+                if not isinstance(results, list):
+                    logger.error("Your on_register_report handler must return a list of tuples or None; "
+                                 f"it returned '{results}' ({results.__class__.__name__}).")
+                    results = None
+                else:
+                    for i, r in enumerate(results):
+                        if r is None:
+                            continue
+                        if not isinstance(r, tuple):
+                            if mode == 'compact':
+                                logger.error("Your on_register_report handler must return a tuple or None; "
+                                             f"it returned '{r}' ({r.__class__.__name__}).")
+                            elif mode == 'full':
+                                logger.error("Your on_register_report handler must return a list of tuples or None; "
+                                             f"The first item from the list was '{r}' ({r.__class__.__name__}).")
+                            results[i] = None
+                    # If we used compact mode, prepend the r_id to each result
+                    # (this is already there when using the full mode)
+                    if mode == 'compact':
+                        results = [(report['report_descriptions'][i]['r_id'], *results[i])
+                                   for i in range(len(report['report_descriptions'])) if isinstance(results[i], tuple)]
+            report_requests.append(results)
+        utils.validate_report_request_tuples(report_requests, mode=mode)
+
+        for i, report_request in enumerate(report_requests):
+            if report_request is None or len(report_request) == 0 or all(rrq is None for rrq in report_request):
+                continue
+            # Check if all sampling rates per report_request are the same
+            sampling_interval = min(rrq[2] for rrq in report_request if isinstance(rrq, tuple))
+            if not all(rrq is not None and report_request[0][2] == sampling_interval for rrq in report_request):
+                logger.error("OpenADR does not support multiple different sampling rates per "
+                             "report. OpenLEADR will set all sampling rates to "
+                             f"{sampling_interval}")
+
+        # Form the report request
+        oadr_report_requests = []
+        for i, report_request in enumerate(report_requests):
+            if report_request is None or len(report_request) == 0 or all(rrq is None for rrq in report_request):
+                continue
+
+            orig_report = payload['reports'][i]
+            report_specifier_id = orig_report['report_specifier_id']
+            report_request_id = utils.generate_id()
+            specifier_payloads = []
+            for rrq in report_request:
+                if len(rrq) == 3:
+                    r_id, callback, sampling_interval = rrq
+                    report_interval = sampling_interval
+                elif len(rrq) == 4:
+                    r_id, callback, sampling_interval, report_interval = rrq
+
+                report_description = utils.find_by(orig_report['report_descriptions'], 'r_id', r_id)
+                reading_type = report_description['reading_type']
+                specifier_payloads.append(objects.SpecifierPayload(r_id=r_id,
+                                                                   reading_type=reading_type))
+                # Append the callback to our list of known callbacks
+                self.report_callbacks[(report_request_id, r_id)] = callback
+
+            # Add the ReportSpecifier to the ReportRequest
+            report_specifier = objects.ReportSpecifier(report_specifier_id=report_specifier_id,
+                                                       granularity=sampling_interval,
+                                                       report_back_duration=report_interval,
+                                                       specifier_payloads=specifier_payloads)
+
+            # Add the ReportRequest to our outgoing message
+            oadr_report_requests.append(objects.ReportRequest(report_request_id=report_request_id,
+                                                              report_specifier=report_specifier))
+
+        # Put the report requests back together
         response_type = 'oadrRegisteredReport'
-        response_payload = {"response": {"response_code": 200,
-                                         "response_description": "OK",
-                                         "request_id": payload['request_id']},
-                            "ven_id": '123'}
+        response_payload = {'report_requests': oadr_report_requests}
         return response_type, response_payload
 
-    @handler('oadrRequestReport')
-    async def request_report(self, payload):
+    async def on_register_report(self, payload):
         """
-        Provide the VEN with the latest report.
+        Pre-handler for a oadrOnRegisterReport message. This will call your own handler (if defined)
+        to allow for requesting the offered reports.
         """
-        print("Called Request Report")
+        logger.warning("You should implement and register your own on_register_report handler "
+                       "if you want to receive reports from a VEN. This handler will receive the "
+                       "following arguments: ven_id, resource_id, measurement, unit, scale, "
+                       "min_sampling_interval, max_sampling_interval and should return either "
+                       "None or (callback, sampling_interval) or "
+                       "(callback, sampling_interval, reporting_interval).")
+        return None
 
     @handler('oadrUpdateReport')
     async def update_report(self, payload):
         """
-        Updates an existing report from this VEN in our database.
+        Handle a report that we received from the VEN.
         """
-        print("Called Update Report")
+        for report in payload['reports']:
+            report_request_id = report['report_request_id']
+            if not self.report_callbacks:
+                result = self.on_update_report(report)
+                if iscoroutine(result):
+                    result = await result
+                continue
+            for r_id, values in utils.group_by(report['intervals'], 'report_payload.r_id').items():
+                # Find the callback that was registered.
+                if (report_request_id, r_id) in self.report_callbacks:
+                    # Collect the values
+                    values = [(ri['dtstart'], ri['report_payload']['value']) for ri in values]
+                    # Call the callback function to deliver the values
+                    result = self.report_callbacks[(report_request_id, r_id)](values)
+                    if iscoroutine(result):
+                        result = await result
+
+        response_type = 'oadrUpdatedReport'
+        response_payload = {}
+        return response_type, response_payload
 
-    @handler('oadrCancelReport')
-    async def cancel_report(self, payload):
+    async def on_update_report(self, payload):
         """
-        Cancels a previously received report from this VEN.
+        Placeholder for the on_update_report handler.
         """
-        print("Called Cancel Report")
+        logger.warning("You should implement and register your own on_update_report handler "
+                       "to deal with reports that your receive from the VEN. This handler will "
+                       "receive either a complete oadrReport dict, or a list of (datetime, value) "
+                       "tuples that you can then process how you see fit. You don't "
+                       "need to return anything from that handler.")
+        return None

openleadr/service/vtn_service.py

@@ -16,19 +16,23 @@
 
 from asyncio import iscoroutine
 from http import HTTPStatus
-import os
+import logging
 
 from aiohttp import web
-from jinja2 import Environment, PackageLoader, select_autoescape
+from lxml.etree import XMLSyntaxError
+from signxml.exceptions import InvalidSignature
 
 from .. import errors
-from ..messaging import create_message, parse_message
-from ..utils import generate_id
+from ..enums import STATUS_CODES
+from ..messaging import parse_message, validate_xml_schema, authenticate_message
+from ..utils import generate_id, get_cert_fingerprint_from_request
 
 from dataclasses import is_dataclass, asdict
 
-class VTNService:
+logger = logging.getLogger('openleadr')
+
 
+class VTNService:
     def __init__(self, vtn_id):
         self.vtn_id = vtn_id
         self.handlers = {}
@@ -40,9 +44,90 @@ class VTNService:
         """
         Handle all incoming POST requests.
         """
-        content = await request.read()
-        message_type, message_payload = self._parse_message(content)
+        try:
+            # Check the Content-Type header
+            content_type = request.headers.get('content-type', '')
+            if not content_type.lower().startswith("application/xml"):
+                raise errors.HTTPError(response_code=HTTPStatus.BAD_REQUEST,
+                                       response_description="The Content-Type header must be application/xml, "
+                                                            "you provided {request.headers.get('content-type', '')}")
+            content = await request.read()
+
+            # Validate the message to the XML Schema
+            message_tree = validate_xml_schema(content)
+
+            # Parse the message to a type and payload dict
+            message_type, message_payload = parse_message(content)
+
+            if 'vtn_id' in message_payload \
+                    and message_payload['vtn_id'] is not None \
+                    and message_payload['vtn_id'] != self.vtn_id:
+                raise errors.InvalidIdError(f"The supplied vtnID is invalid. It should be '{self.vtn_id}', "
+                                            f"you supplied {message_payload['vtn_id']}.")
+
+            # Authenticate the message
+            if request.secure and 'ven_id' in message_payload:
+                await authenticate_message(request, message_tree, message_payload,
+                                           self.fingerprint_lookup)
+
+            # Pass the message off to the handler and get the response type and payload
+            try:
+                # Add the request fingerprint to the message so that the handler can check for it.
+                if request.secure and message_type == 'oadrCreatePartyRegistration':
+                    message_payload['fingerprint'] = get_cert_fingerprint_from_request(request)
+                response_type, response_payload = await self.handle_message(message_type,
+                                                                            message_payload)
+            except Exception as err:
+                logger.error("An exception occurred during the execution of your "
+                             f"{self.__class__.__name__} handler: "
+                             f"{err.__class__.__name__}: {err}")
+                raise err
+
+            if 'response' not in response_payload:
+                response_payload['response'] = {'response_code': 200,
+                                                'response_description': 'OK',
+                                                'request_id': message_payload.get('request_id')}
+            response_payload['vtn_id'] = self.vtn_id
+            if 'ven_id' not in response_payload:
+                response_payload['ven_id'] = message_payload.get('ven_id')
+        except errors.ProtocolError as err:
+            # In case of an OpenADR error, return a valid OpenADR message
+            response_type, response_payload = self.error_response(message_type,
+                                                                  err.response_code,
+                                                                  err.response_description)
+            msg = self._create_message(response_type, **response_payload)
+            response = web.Response(text=msg,
+                                    status=HTTPStatus.OK,
+                                    content_type='application/xml')
+        except errors.HTTPError as err:
+            # If we throw a http-related error, deal with it here
+            response = web.Response(text=err.response_description,
+                                    status=err.response_code)
+        except XMLSyntaxError as err:
+            logger.warning(f"XML schema validation of incoming message failed: {err}.")
+            response = web.Response(text=f'XML failed validation: {err}',
+                                    status=HTTPStatus.BAD_REQUEST)
+        except errors.FingerprintMismatch as err:
+            logger.warning(err)
+            response = web.Response(text=str(err),
+                                    status=HTTPStatus.FORBIDDEN)
+        except InvalidSignature:
+            logger.warning("Incoming message had invalid signature, ignoring.")
+            response = web.Response(text='Invalid Signature',
+                                    status=HTTPStatus.FORBIDDEN)
+        except Exception as err:
+            # In case of some other error, return a HTTP 500
+            logger.error(f"The VTN server encountered an error: {err.__class__.__name__}: {err}")
+            response = web.Response(status=HTTPStatus.INTERNAL_SERVER_ERROR)
+        else:
+            # We've successfully handled this message
+            msg = self._create_message(response_type, **response_payload)
+            response = web.Response(text=msg,
+                                    status=HTTPStatus.OK,
+                                    content_type='application/xml')
+        return response
 
+    async def handle_message(self, message_type, message_payload):
         if message_type in self.handlers:
             handler = self.handlers[message_type]
             result = handler(message_payload)
@@ -52,6 +137,8 @@ class VTNService:
                 response_type, response_payload = result
                 if is_dataclass(response_payload):
                     response_payload = asdict(response_payload)
+                elif response_payload is None:
+                    response_payload = {}
             else:
                 response_type, response_payload = 'oadrResponse', {}
 
@@ -64,20 +151,22 @@ class VTNService:
                                             'response_description': 'OK'}
             response_payload['request_id'] = generate_id()
 
-            # Create the XML response
-            msg = self._create_message(response_type, **response_payload)
-            response = web.Response(text=msg,
-                                    status=HTTPStatus.OK,
-                                    content_type='application/xml')
-
         else:
-            msg = self._create_message('oadrResponse',
-                                       ven_id=message_payload.get('ven_id'),
-                                       status_code=errorcodes.COMPLIANCE_ERROR,
-                                       status_description=f"A message of type {message_type} should not be sent to this endpoint")
-
-            response = web.Response(
-                text=msg,
-                status=HTTPStatus.BAD_REQUEST,
-                content_type='application/xml')
-        return response
+            response_type, response_payload = self.error_response('oadrResponse',
+                                                                  STATUS_CODES.COMPLIANCE_ERROR,
+                                                                  "A message of type "
+                                                                  f"{message_type} should not be "
+                                                                  "sent to this endpoint")
+        logger.info(f"Responding to {message_type} with a {response_type} message: {response_payload}.")
+        return response_type, response_payload
+
+    def error_response(self, message_type, error_code, error_description):
+        if message_type == 'oadrCreatePartyRegistration':
+            response_type = 'oadrCreatedPartyRegistration'
+        if message_type == 'oadrRequestEvent':
+            response_type = 'oadrDistributeEvent'
+        else:
+            response_type = 'oadrResponse'
+        response_payload = {'response': {'response_code': error_code,
+                                         'response_description': error_description}}
+        return response_type, response_payload

openleadr/templates/oadrCreatedEvent.xml

@@ -8,7 +8,7 @@
         <requestID xmlns="http://docs.oasis-open.org/ns/energyinterop/201110/payloads">{{ response.request_id }}</requestID>
         {% endif %}
       </ei:eiResponse>
-      {% if event_responses is defined and event_response is not none %}
+      {% if event_responses is defined and event_responses is not none %}
       <ei:eventResponses>
         {% for event_response in event_responses %}
         <ei:eventResponse>

openleadr/templates/oadrCreatedPartyRegistration.xml

@@ -25,7 +25,7 @@
         <oadr:oadrTransports>
           {% for transport in profile.transports %}
           <oadr:oadrTransport>
-            <oadr:oadrTransportName>simpleHttp</oadr:oadrTransportName>
+            <oadr:oadrTransportName>{{ transport.transport_name }}</oadr:oadrTransportName>
           </oadr:oadrTransport>
           {% endfor %}
         </oadr:oadrTransports>
@@ -38,4 +38,4 @@
     </oadr:oadrRequestedOadrPollFreq>
     {% endif %}
   </oadr:oadrCreatedPartyRegistration>
-</oadr:oadrSignedObject>
+</oadr:oadrSignedObject>

openleadr/templates/oadrPayload.xml

@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="utf-8"?>
-<oadrPayload xmlns="http://openadr.org/oadr-2.0b/2012/07">
+<oadr:oadrPayload xmlns:oadr="http://openadr.org/oadr-2.0b/2012/07">
 {% if signature: %}{{ signature|safe }}{% endif %}
 {{ signed_object|safe }}
-</oadrPayload>
+</oadr:oadrPayload>

openleadr/templates/oadrPoll.xml

@@ -1,4 +1,4 @@
-<oadr:oadrSignedObject xmlns:oadr="http://openadr.org/oadr-2.0b/2012/07" oadr:Id="oadrSignedObject" xmlns:xs="http://www.w3.org/2001/XMLSchema">
+<oadr:oadrSignedObject xmlns:oadr="http://openadr.org/oadr-2.0b/2012/07" oadr:Id="oadrSignedObject">
   <oadr:oadrPoll ei:schemaVersion="2.0b" xmlns:ei="http://docs.oasis-open.org/ns/energyinterop/201110">
     <ei:venID>{{ ven_id }}</ei:venID>
   </oadr:oadrPoll>

openleadr/templates/oadrRegisterReport.xml

@@ -14,10 +14,14 @@
       </xcal:duration>
       {% endif %}
       <ei:eiReportID>{{ report.report_id }}</ei:eiReportID>
-  {% for r_id, report_description in report.report_descriptions.items() %}
+  {% for report_description in report.report_descriptions %}
       {% include 'parts/oadrReportDescription.xml' %}
   {% endfor %}
+      {% if report.report_request_id is defined and report.report_request_id is not none %}
       <ei:reportRequestID>{{ report.report_request_id }}</ei:reportRequestID>
+      {% else %}
+      <ei:reportRequestID></ei:reportRequestID>
+      {% endif %}
       <ei:reportSpecifierID>{{ report.report_specifier_id }}</ei:reportSpecifierID>
       <ei:reportName>{{ report.report_name }}</ei:reportName>
       <ei:createdDateTime>{{ report.created_date_time|datetimeformat }}</ei:createdDateTime>

openleadr/templates/oadrRegisteredReport.xml

@@ -8,6 +8,8 @@
     {% for report_request in report_requests %}
     {% include 'parts/oadrReportRequest.xml' %}
     {% endfor %}
+    {% if ven_id is defined and ven_id is not none %}
     <ei:venID>{{ ven_id }}</ei:venID>
+    {% endif %}
   </oadr:oadrRegisteredReport>
 </oadr:oadrSignedObject>

openleadr/templates/oadrUpdateReport.xml

@@ -3,19 +3,52 @@
     <pyld:requestID>{{ request_id }}</pyld:requestID>
     {% if reports %}
     {% for report in reports %}
-    <oadr:oadrReport>
+    <oadr:oadrReport xmlns:xcal="urn:ietf:params:xml:ns:icalendar-2.0">
+      {% if report.dtstart is defined and report.dtstart is not none %}
+      <xcal:dtstart>
+        <xcal:date-time>{{ report.dtstart|datetimeformat }}</xcal:date-time>
+      </xcal:dtstart>
+      {% endif %}
+
+      {% if report.intervals %}
+      <strm:intervals xmlns:strm="urn:ietf:params:xml:ns:icalendar-2.0:stream">
+        {% for interval in report.intervals %}
+        <ei:interval>
+          <xcal:dtstart>
+            <xcal:date-time>{{ interval.dtstart|datetimeformat }}</xcal:date-time>
+          </xcal:dtstart>
+          <oadr:oadrReportPayload>
+            <ei:rID>{{ interval.report_payload.r_id }}</ei:rID>
+            {% if interval.report_payload.confidence is defined and interval.report_payload.confidence is not none %}
+            <ei:confidence>{{ interval.report_payload.confidence }}</ei:confidence>
+            {% endif %}
+            {% if interval.report_payload.accuracy is defined and interval.report_payload.accuracy is not none %}
+            <ei:accuracy>{{ interval.report_payload.accuracy }}</ei:accuracy>
+            {% endif %}
+            <ei:payloadFloat>
+              <ei:value>{{ interval.report_payload.value }}</ei:value>
+            </ei:payloadFloat>
+            {% if interval.report_payload.data_quality is defined and interval.report_payload.data_quality is not none %}
+            <oadr:oadrDataQuality>{{ interval.report_payload.data_quality }}</oadr:oadrDataQuality>
+            {% endif %}
+          </oadr:oadrReportPayload>
+        </ei:interval>
+        {% endfor %}
+      </strm:intervals>
+      {% endif %}
+
       <ei:eiReportID>{{ report.report_id }}</ei:eiReportID>
       {% if report.report_descriptions %}
-      {% for r_id, report_description in report.report_descriptions.items() %}
+      {% for report_description in report.report_descriptions %}
       {% include 'parts/oadrReportDescription.xml' %}
+      {% endfor %}
+      {% endif %}
       <ei:reportRequestID>{{ report.report_request_id }}</ei:reportRequestID>
       <ei:reportSpecifierID>{{ report.report_specifier_id }}</ei:reportSpecifierID>
       {% if report.report_name %}
       <ei:reportName>{{ report.report_name }}</ei:reportName>
       {% endif %}
       <ei:createdDateTime>{{ report.created_date_time|datetimeformat }}</ei:createdDateTime>
-      {% endfor %}
-      {% endif %}
     </oadr:oadrReport>
     {% endfor %}
     {% endif %}
@@ -23,4 +56,4 @@
     <ei:venID>{{ ven_id }}</ei:venID>
     {% endif %}
   </oadr:oadrUpdateReport>
-</oadr:oadrSignedObject>
+</oadr:oadrSignedObject>

openleadr/templates/parts/eiActivePeriod.xml

@@ -13,19 +13,19 @@
             </tolerate>
         </tolerance>
         {% endif %}
-        {% if event.active_period.notification %}
+        {% if event.active_period.notification_period %}
         <ei:x-eiNotification>
-            <duration>{{ event.active_period.notification|timedeltaformat }}</duration>
+            <duration>{{ event.active_period.notification_period|timedeltaformat }}</duration>
         </ei:x-eiNotification>
         {% endif %}
-        {% if event.active_period.ramp_up %}
+        {% if event.active_period.ramp_up_period %}
         <ei:x-eiRampUp>
-            <duration>{{ event.active_period.ramp_up|timedeltaformat }}</duration>
+            <duration>{{ event.active_period.ramp_up_period|timedeltaformat }}</duration>
         </ei:x-eiRampUp>
         {% endif %}
-        {% if event.active_period.recovery %}
+        {% if event.active_period.recovery_period %}
         <ei:x-eiRecovery>
-            <duration>{{ event.active_period.recovery|timedeltaformat }}</duration>
+            <duration>{{ event.active_period.recovery_period|timedeltaformat }}</duration>
         </ei:x-eiRecovery>
         {% endif %}
     </properties>

openleadr/templates/parts/eiEvent.xml

@@ -3,7 +3,7 @@
         {% include 'parts/eiEventDescriptor.xml' %}
         {% include 'parts/eiActivePeriod.xml' %}
         <ei:eiEventSignals>
-            {% for signal in event.event_signals %}
+            {% for event_signal in event.event_signals %}
                 {% include 'parts/eiEventSignal.xml' %}
             {% endfor %}
         </ei:eiEventSignals>

openleadr/templates/parts/eiEventSignal.xml

@@ -1,10 +1,17 @@
 <ei:eiEventSignal>
-    <intervals xmlns="urn:ietf:params:xml:ns:icalendar-2.0:stream">
-    {% for interval in signal.intervals %}
+    <strm:intervals xmlns:strm="urn:ietf:params:xml:ns:icalendar-2.0:stream" xmlns:xcal="urn:ietf:params:xml:ns:icalendar-2.0">
+    {% for interval in event_signal.intervals %}
         <ei:interval>
-            <duration xmlns="urn:ietf:params:xml:ns:icalendar-2.0">
-                <duration>{{ interval.duration|timedeltaformat }}</duration>
-            </duration>
+            {% if interval.dtstart is defined and interval.dtstart is not none %}
+            <xcal:dtstart>
+                <xcal:date-time>{{ interval.dtstart|datetimeformat }}</xcal:date-time>
+            </xcal:dtstart>
+            {% endif %}
+            {% if interval.duration is defined and interval.duration is not none %}
+            <xcal:duration>
+                <xcal:duration>{{ interval.duration|timedeltaformat }}</xcal:duration>
+            </xcal:duration>
+            {% endif %}
             <uid xmlns="urn:ietf:params:xml:ns:icalendar-2.0">
                 <text>{{ loop.index0 }}</text>
             </uid>
@@ -15,14 +22,15 @@
             </ei:signalPayload>
         </ei:interval>
     {% endfor %}
-    </intervals>
-    <ei:signalName>{{ signal.signal_name }}</ei:signalName>
-    <ei:signalType>{{ signal.signal_type }}</ei:signalType>
-    <ei:signalID>{{ signal.signal_id }}</ei:signalID>
-    {% if signal.current_value is defined and signal.current_value is not none %}
+    </strm:intervals>
+    <ei:signalName>{{ event_signal.signal_name }}</ei:signalName>
+    <ei:signalType>{{ event_signal.signal_type }}</ei:signalType>
+    <ei:signalID>{{ event_signal.signal_id }}</ei:signalID>
+    {% include 'parts/eventSignalEmix.xml' %}
+    {% if event_signal.current_value is defined and event_signal.current_value is not none %}
     <ei:currentValue>
         <ei:payloadFloat>
-            <ei:value>{{ signal.current_value }}</ei:value>
+            <ei:value>{{ event_signal.current_value }}</ei:value>
         </ei:payloadFloat>
     </ei:currentValue>
     {% endif %}

openleadr/templates/parts/eiTarget.xml

@@ -1,27 +1,27 @@
 <ei:eiTarget>
-    {% if target.emix_interfaces %}
+    {% if target.emix_interfaces is defined and target.emix_interfaces is not none %}
         {% for emix_interface in target.emix_interface %}
             {% include 'parts/emixInterface.xml' %}
         {% endfor %}
     {% endif %}
 
-    {% if target.group_id %}
+    {% if target.group_id is defined and target.group_id is not none %}
         <ei:groupID>{{ target.group_id }}</ei:groupID>
     {% endif %}
 
-    {% if target.group_name %}
+    {% if target.group_name is defined and target.group_name is not none %}
         <ei:groupName>{{ target.group_name }}</ei:groupName>
     {% endif %}
 
-    {% if target.resource_id %}
+    {% if target.resource_id is defined and target.resource_id is not none %}
         <ei:resourceID>{{ target.resource_id }}</ei:resourceID>
     {% endif %}
 
-    {% if target.ven_id %}
+    {% if target.ven_id is defined and target.ven_id is not none %}
         <ei:venID>{{ target.ven_id }}</ei:venID>
     {% endif %}
 
-    {% if target.party_id %}
+    {% if target.party_id is defined and target.party_id is not none %}
         <ei:partyID>{{ target.party_id }}</ei:partyID>
     {% endif %}
 </ei:eiTarget>

openleadr/templates/parts/eventSignalEmix.xml

@@ -0,0 +1,16 @@
+  {% if event_signal.measurement is defined and event_signal.measurement is not none %}
+  <{{ event_signal.measurement.ns }}:{{ event_signal.measurement.name }} xmlns:scale="http://docs.oasis-open.org/ns/emix/2011/06/siscale" xmlns:power="http://docs.oasis-open.org/ns/emix/2011/06/power" >
+    <{{ event_signal.measurement.ns }}:itemDescription>{{ event_signal.measurement.description }}</{{ event_signal.measurement.ns }}:itemDescription>
+    <{{ event_signal.measurement.ns }}:itemUnits>{{ event_signal.measurement.unit }}</{{ event_signal.measurement.ns }}:itemUnits>
+    {% if event_signal.measurement.pulse_factor %}<oadr:pulseFactor>{{ event_signal.measurement.pulse_factor }}</oadr:pulseFactor>{% else %}
+    <scale:siScaleCode>{{ event_signal.measurement.scale }}</scale:siScaleCode>
+    {% endif %}
+    {% if event_signal.measurement.power_attributes %}
+    <power:powerAttributes>
+      <power:hertz>{{ event_signal.measurement.power_attributes.hertz }}</power:hertz>
+      <power:voltage>{{ event_signal.measurement.power_attributes.voltage }}</power:voltage>
+      <power:ac>{{ event_signal.measurement.power_attributes.ac|booleanformat }}</power:ac>
+    </power:powerAttributes>
+    {% endif %}
+  </{{ event_signal.measurement.ns }}:{{ event_signal.measurement.name }}>
+  {% endif %}

openleadr/templates/parts/oadrReportDescription.xml

@@ -1,85 +1,40 @@
-<oadr:oadrReportDescription xmlns:emix="http://docs.oasis-open.org/ns/emix/2011/06">
-  <ei:rID>{{ r_id }}</ei:rID>
-  {% if report_description.report_subjects %}
+<oadr:oadrReportDescription xmlns:emix="http://docs.oasis-open.org/ns/emix/2011/06" xmlns:power="http://docs.oasis-open.org/ns/emix/2011/06/power">
+  <ei:rID>{{ report_description.r_id }}</ei:rID>
+  {% if report_description.report_subject and report_description.report_subject.end_device_asset.mrid %}
   <ei:reportSubject>
-  {% for target in report_description.report_subjects %}
-    {% if target.emix_interfaces %}
-        {% for emix_interface in target.emix_interface %}
-            {% include 'parts/emixInterface.xml' %}
-        {% endfor %}
-    {% endif %}
-  {% endfor %}
-
-  {% for target in report_description.report_subjects %}
-    {% if target.group_id %}
-        <ei:groupID>{{ target.group_id }}</ei:groupID>
-    {% endif %}
-  {% endfor %}
-
-  {% for target in report_description.report_subjects %}
-    {% if target.group_name %}
-        <ei:groupName>{{ target.group_name }}</ei:groupName>
-    {% endif %}
-  {% endfor %}
-
-  {% for target in report_description.report_subjects %}
-    {% if target.resource_id %}
-        <ei:resourceID>{{ target.resource_id }}</ei:resourceID>
-    {% endif %}
-  {% endfor %}
-
-  {% for target in report_description.report_subjects %}
-    {% if target.ven_id %}
-        <ei:venID>{{ target.ven_id }}</ei:venID>
-    {% endif %}
-  {% endfor %}
-
-  {% for target in report_description.report_subjects %}
-    {% if target.party_id %}
-        <ei:partyID>{{ target.party_id }}</ei:partyID>
-    {% endif %}
-  {% endfor %}
+    <power:endDeviceAsset>
+      <power:mrid>{{ report_description.report_subject.end_device_asset.mrid }}</power:mrid>
+    </power:endDeviceAsset>
   </ei:reportSubject>
   {% endif %}
-  {% if report_description.report_data_sources %}
+
+  {% if report_description.report_data_source %}
   <ei:reportDataSource>
-  {% for target in report_description.report_data_sources %}
-    {% if target.emix_interfaces %}
+    {% if report_description.report_data_source.emix_interfaces %}
         {% for emix_interface in target.emix_interface %}
             {% include 'parts/emixInterface.xml' %}
         {% endfor %}
     {% endif %}
-  {% endfor %}
 
-  {% for target in report_description.report_data_sources %}
-    {% if target.group_id %}
-        <ei:groupID>{{ target.group_id }}</ei:groupID>
+    {% if report_description.report_data_source.group_id %}
+        <ei:groupID>{{ report_description.report_data_source.group_id }}</ei:groupID>
     {% endif %}
-  {% endfor %}
 
-  {% for target in report_description.report_data_sources %}
-    {% if target.group_name %}
-        <ei:groupName>{{ target.group_name }}</ei:groupName>
+    {% if report_description.report_data_source.group_name %}
+        <ei:groupName>{{ report_description.report_data_source.group_name }}</ei:groupName>
     {% endif %}
-  {% endfor %}
 
-  {% for target in report_description.report_data_sources %}
-    {% if target.resource_id %}
-        <ei:resourceID>{{ target.resource_id }}</ei:resourceID>
+    {% if report_description.report_data_source.resource_id %}
+        <ei:resourceID>{{ report_description.report_data_source.resource_id }}</ei:resourceID>
     {% endif %}
-  {% endfor %}
 
-  {% for target in report_description.report_data_sources %}
-    {% if target.ven_id %}
-        <ei:venID>{{ target.ven_id }}</ei:venID>
+    {% if report_description.report_data_source.ven_id %}
+        <ei:venID>{{ report_description.report_data_source.ven_id }}</ei:venID>
     {% endif %}
-  {% endfor %}
 
-  {% for target in report_description.report_data_sources %}
-    {% if target.party_id %}
-        <ei:partyID>{{ target.party_id }}</ei:partyID>
+    {% if report_description.report_data_source.party_id %}
+        <ei:partyID>{{ report_description.report_data_source.party_id }}</ei:partyID>
     {% endif %}
-  {% endfor %}
   </ei:reportDataSource>
   {% endif %}
   <ei:reportType>{{ report_description.report_type }}</ei:reportType>

openleadr/templates/parts/oadrReportRequest.xml

@@ -5,10 +5,12 @@
     <xcal:granularity>
       <xcal:duration>{{ report_request.report_specifier.granularity|timedeltaformat }}</xcal:duration>
     </xcal:granularity>
+    {% if report_request.report_specifier.report_back_duration is defined and report_request.report_specifier.report_back_duration is not none %}
     <ei:reportBackDuration>
       <xcal:duration>{{ report_request.report_specifier.report_back_duration|timedeltaformat }}</xcal:duration>
     </ei:reportBackDuration>
-    {% if report_request.report_specifier.report_interval %}
+    {% endif %}
+    {% if report_request.report_specifier.report_interval is defined and report_request.report_specifier.report_interval is not none %}
     <ei:reportInterval>
       <xcal:properties>
         <xcal:dtstart>
@@ -17,34 +19,14 @@
         <xcal:duration>
           <xcal:duration>{{ report_request.report_specifier.report_interval.duration|timedeltaformat }}</xcal:duration>
         </xcal:duration>
-        {% if report_request.report_specifier.report_interval.tolerance %}
-        <xcal:tolerance>
-          <xcal:tolerate>
-            <xcal:startafter>{{ report_request.report_specifier.report_interval.tolerance.tolerate.startafter|timedeltaformat }}</xcal:startafter>
-          </xcal:tolerate>
-        </xcal:tolerance>
-        {% endif %}
-        {% if report_request.report_specifier.report_interval.notification %}
-        <ei:x-eiNotification>
-          <xcal:duration>{{ report_request.report_specifier.report_interval.notification|timedeltaformat }}</xcal:duration>
-        </ei:x-eiNotification>
-        {% endif %}
-        {% if report_request.report_specifier.report_interval.ramp_up %}
-        <ei:x-eiRampUp>
-          <xcal:duration>{{ report_request.report_specifier.report_interval.ramp_up|timedeltaformat }}</xcal:duration>
-        </ei:x-eiRampUp>
-        {% endif %}
-        {% if report_request.report_specifier.report_interval.recovery %}
-        <ei:x-eiRecovery>
-          <xcal:duration>{{ report_request.report_specifier.report_interval.recovery|timedeltaformat }}</xcal:duration>
-        </ei:x-eiRecovery>
-        {% endif %}
       </xcal:properties>
     </ei:reportInterval>
     {% endif %}
+    {% for specifier_payload in report_request.report_specifier.specifier_payloads %}
     <ei:specifierPayload>
-      <ei:rID>{{ report_request.report_specifier.specifier_payload.r_id }}</ei:rID>
-      <ei:readingType>{{ report_request.report_specifier.specifier_payload.reading_type }}</ei:readingType>
+      <ei:rID>{{ specifier_payload.r_id }}</ei:rID>
+      <ei:readingType>{{ specifier_payload.reading_type }}</ei:readingType>
     </ei:specifierPayload>
+    {% endfor %}
   </ei:reportSpecifier>
 </oadr:oadrReportRequest>

openleadr/templates/parts/reportDescriptionEmix.xml

@@ -1,81 +1,16 @@
-  {% if report_description.power_real %}
-  <powerReal xmlns="http://docs.oasis-open.org/ns/emix/2011/06/power" xmlns:scale="http://docs.oasis-open.org/ns/emix/2011/06/siscale">
-    <itemDescription>{{ report_description.power_real.item_description }}</itemDescription>
-    <itemUnits>{{ report_description.power_real.item_units }}</itemUnits>
-    <scale:siScaleCode>{{ report_description.power_real.si_scale_code }}</scale:siScaleCode>
-    <powerAttributes>
-      <hertz>{{ report_description.power_real.power_attributes.hertz }}</hertz>
-      <voltage>{{ report_description.power_real.power_attributes.voltage }}</voltage>
-      <ac>{{ report_description.power_real.power_attributes.ac|booleanformat }}</ac>
-    </powerAttributes>
-  </powerReal>
-  {% endif %}
-
-  {% if report_description.power_apparent %}
-  <powerApparent xmlns="http://docs.oasis-open.org/ns/emix/2011/06/power" xmlns:scale="http://docs.oasis-open.org/ns/emix/2011/06/siscale">
-    <itemDescription>{{ report_description.power_apparent.item_description }}</itemDescription>
-    <itemUnits>{{ report_description.power_apparent.item_units }}</itemUnits>
-    <scale:siScaleCode>{{ report_description.power_apparent.si_scale_code }}</scale:siScaleCode>
-    <powerAttributes>
-      <hertz>{{ report_description.power_apparent.power_attributes.hertz }}</hertz>
-      <voltage>{{ report_description.power_apparent.power_attributes.voltage }}</voltage>
-      <ac>{{ report_description.power_apparent.power_attributes.ac|booleanformat }}</ac>
-    </powerAttributes>
-  </powerApparent>
-  {% endif %}
-
-  {% if report_description.power_reactive %}
-  <powerReactive xmlns="http://docs.oasis-open.org/ns/emix/2011/06/power" xmlns:scale="http://docs.oasis-open.org/ns/emix/2011/06/siscale">
-    <itemDescription>{{ report_description.power_reactive.item_description }}</itemDescription>
-    <itemUnits>{{ report_description.power_reactive.item_units }}</itemUnits>
-    <scale:siScaleCode>{{ report_description.power_reactive.si_scale_code }}</scale:siScaleCode>
-    <powerAttributes>
-      <hertz>{{ report_description.power_reactive.power_attributes.hertz }}</hertz>
-      <voltage>{{ report_description.power_reactive.power_attributes.voltage }}</voltage>
-      <ac>{{ report_description.power_reactive.power_attributes.ac|booleanformat }}</ac>
-    </powerAttributes>
-  </powerReactive>
-  {% endif %}
-
-  {% if report_description.energy_real %}
-  <energyReal xmlns="http://docs.oasis-open.org/ns/emix/2011/06/power" xmlns:scale="http://docs.oasis-open.org/ns/emix/2011/06/siscale">
-    <itemDescription>{{ report_description.energy_real.item_description }}</itemDescription>
-    <itemUnits>{{ report_description.energy_real.item_units }}</itemUnits>
-    <scale:siScaleCode>{{ report_description.energy_real.si_scale_code }}</scale:siScaleCode>
-  </energyReal>
-  {% endif %}
-
-  {% if report_description.energy_apparent %}
-  <energyApparent xmlns="http://docs.oasis-open.org/ns/emix/2011/06/power" xmlns:scale="http://docs.oasis-open.org/ns/emix/2011/06/siscale">
-    <itemDescription>{{ report_description.energy_apparent.item_description }}</itemDescription>
-    <itemUnits>{{ report_description.energy_apparent.item_units }}</itemUnits>
-    <scale:siScaleCode>{{ report_description.energy_apparent.si_scale_code }}</scale:siScaleCode>
-  </energyApparent>
-  {% endif %}
-
-  {% if report_description.energy_reactive %}
-  <energyReactive xmlns="http://docs.oasis-open.org/ns/emix/2011/06/power" xmlns:scale="http://docs.oasis-open.org/ns/emix/2011/06/siscale">
-    <itemDescription>{{ report_description.energy_reactive.item_description }}</itemDescription>
-    <itemUnits>{{ report_description.energy_reactive.item_units }}</itemUnits>
-    <scale:siScaleCode>{{ report_description.energy_reactive.si_scale_code }}</scale:siScaleCode>
-  </energyReactive>
-  {% endif %}
-
-  {% if report_description.energy_quantity %}
-  <energyQuantity xmlns="http://docs.oasis-open.org/ns/emix/2011/06/power" xmlns:scale="http://docs.oasis-open.org/ns/emix/2011/06/siscale">
-    <quantity>{{ report_description.energy_quantity.quantity }}</quantity>
-    <energyItem xmlns="http://docs.oasis-open.org/ns/emix/2011/06/power" xmlns:scale="http://docs.oasis-open.org/ns/emix/2011/06/siscale">
-      <itemDescription>{{ report_description.energy_quantity.energy_item.item_description }}</itemDescription>
-      <itemUnits>{{ report_description.energy_quantity.energy_item.item_units }}</itemUnits>
-      <scale:siScaleCode>{{ report_description.energy_quantity.energy_item.si_scale_code }}</scale:siScaleCode>
-    </energyItem>
-  </energyQuantity>
-  {% endif %}
-
-  {% if report_description.voltage %}
-  <voltage xmlns="http://docs.oasis-open.org/ns/emix/2011/06/power" xmlns:scale="http://docs.oasis-open.org/ns/emix/2011/06/siscale">
-    <itemDescription>{{ report_description.voltage.item_description }}</itemDescription>
-    <itemUnits>{{ report_description.voltage.item_units }}</itemUnits>
-    <scale:siScaleCode>{{ report_description.voltage.si_scale_code }}</scale:siScaleCode>
-  </voltage>
+  {% if report_description.measurement is defined and report_description.measurement is not none %}
+  <{{ report_description.measurement.ns }}:{{ report_description.measurement.name }} xmlns:scale="http://docs.oasis-open.org/ns/emix/2011/06/siscale" xmlns:power="http://docs.oasis-open.org/ns/emix/2011/06/power" >
+    <{{ report_description.measurement.ns }}:itemDescription>{{ report_description.measurement.description }}</{{ report_description.measurement.ns }}:itemDescription>
+    <{{ report_description.measurement.ns }}:itemUnits>{{ report_description.measurement.unit }}</{{ report_description.measurement.ns }}:itemUnits>
+    {% if report_description.measurement.pulse_factor %}<oadr:pulseFactor>{{ report_description.measurement.pulse_factor }}</oadr:pulseFactor>{% else %}
+    <scale:siScaleCode>{{ report_description.measurement.scale }}</scale:siScaleCode>
+    {% endif %}
+    {% if report_description.measurement.power_attributes %}
+    <power:powerAttributes>
+      <power:hertz>{{ report_description.measurement.power_attributes.hertz }}</power:hertz>
+      <power:voltage>{{ report_description.measurement.power_attributes.voltage }}</power:voltage>
+      <power:ac>{{ report_description.measurement.power_attributes.ac|booleanformat }}</power:ac>
+    </power:powerAttributes>
+    {% endif %}
+  </{{ report_description.measurement.ns }}:{{ report_description.measurement.name }}>
   {% endif %}

openleadr/utils.py

@@ -14,45 +14,29 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from asyncio import iscoroutine
 from datetime import datetime, timedelta, timezone
 from dataclasses import is_dataclass, asdict
-import random
-import string
 from collections import OrderedDict
-import itertools
+from openleadr import enums, objects
+import asyncio
 import re
 import ssl
 import hashlib
 import uuid
+import logging
 
-from openleadr import config
+logger = logging.getLogger('openleadr')
 
 DATETIME_FORMAT = "%Y-%m-%dT%H:%M:%S.%fZ"
 DATETIME_FORMAT_NO_MICROSECONDS = "%Y-%m-%dT%H:%M:%SZ"
 
+
 def generate_id(*args, **kwargs):
     """
     Generate a string that can be used as an identifier in OpenADR messages.
     """
     return str(uuid.uuid4())
 
-def indent_xml(message):
-    """
-    Indents the XML in a nice way.
-    """
-    INDENT_SIZE = 2
-    lines = [line.strip() for line in message.split("\n") if line.strip() != ""]
-    indent = 0
-    for i, line in enumerate(lines):
-        if i == 0:
-            continue
-        if re.search(r'^</[^>]+>$', line):
-            indent = indent - INDENT_SIZE
-        lines[i] = " " * indent + line
-        if not (re.search(r'</[^>]+>$', line) or line.endswith("/>")):
-            indent = indent + INDENT_SIZE
-    return "\n".join(lines)
 
 def flatten_xml(message):
     """
@@ -64,6 +48,7 @@ def flatten_xml(message):
         line = re.sub(r'\s\s+', ' ', line)
     return "".join(lines)
 
+
 def normalize_dict(ordered_dict):
     """
     Main conversion function for the output of xmltodict to the OpenLEADR
@@ -79,6 +64,9 @@ def normalize_dict(ordered_dict):
             key = key[4:]
         elif key.startswith('ei'):
             key = key[2:]
+        # Don't normalize the measurement descriptions
+        if key in enums._MEASUREMENT_NAMESPACES:
+            return key
         key = re.sub(r'([a-z])([A-Z])', r'\1_\2', key)
         if '-' in key:
             key = key.replace('-', '_')
@@ -93,6 +81,7 @@ def normalize_dict(ordered_dict):
 
         if isinstance(value, (OrderedDict, dict)):
             d[key] = normalize_dict(value)
+
         elif isinstance(value, list):
             d[key] = []
             for item in value:
@@ -108,7 +97,12 @@ def normalize_dict(ordered_dict):
         elif value in ('true', 'false'):
             d[key] = parse_boolean(value)
         elif isinstance(value, str):
-            d[key] = parse_int(value) or parse_float(value) or value
+            if re.match(r'^-?\d+$', value):
+                d[key] = int(value)
+            elif re.match(r'^-?[\d.]+$', value):
+                d[key] = float(value)
+            else:
+                d[key] = value
         else:
             d[key] = value
 
@@ -118,7 +112,7 @@ def normalize_dict(ordered_dict):
             key = key[5:]
 
         # Group all targets as a list of dicts under the key "target"
-        if key in ("target", "report_subject", "report_data_source"):
+        if key == 'target':
             targets = d.pop(key)
             new_targets = []
             if targets:
@@ -130,17 +124,16 @@ def normalize_dict(ordered_dict):
             d[key + "s"] = new_targets
             key = key + "s"
 
-        # Dig up the properties inside some specific target identifiers
-        # if key in ("aggregated_pnode", "pnode", "service_delivery_point"):
-        #     d[key] = d[key]["node"]
-
-        # if key in ("end_device_asset", "meter_asset"):
-        #     d[key] = d[key]["mrid"]
+            # Also add a targets_by_type element to this dict
+            # to access the targets in a more convenient way.
+            d['targets_by_type'] = group_targets_by_type(new_targets)
 
         # Group all reports as a list of dicts under the key "pending_reports"
         if key == "pending_reports":
-            if isinstance(d[key], dict) and 'report_request_id' in d[key] and isinstance(d[key]['report_request_id'], list):
-                d['pending_reports'] = [{'request_id': rrid} for rrid in d['pending_reports']['report_request_id']]
+            if isinstance(d[key], dict) and 'report_request_id' in d[key] \
+               and isinstance(d[key]['report_request_id'], list):
+                d['pending_reports'] = [{'request_id': rrid}
+                                        for rrid in d['pending_reports']['report_request_id']]
 
         # Group all events al a list of dicts under the key "events"
         elif key == "event" and isinstance(d[key], list):
@@ -163,24 +156,33 @@ def normalize_dict(ordered_dict):
             d = d[key]
 
         # Plurarize some lists
-        elif key in ('report_request', 'report'):
+        elif key in ('report_request', 'report', 'specifier_payload'):
             if isinstance(d[key], list):
                 d[key + 's'] = d.pop(key)
             else:
                 d[key + 's'] = [d.pop(key)]
 
-        elif key == 'report_description':
-            if isinstance(d[key], list):
-                original_descriptions = d.pop(key)
-                report_descriptions = {}
-                for item in original_descriptions:
-                    r_id = item.pop('r_id')
-                    report_descriptions[r_id] = item
-                d[key + 's'] = report_descriptions
-            else:
-                original_description = d.pop(key)
-                r_id = original_description.pop('r_id')
-                d[key + 's'] = {r_id: original_description}
+        elif key in ('report_description', 'event_signal'):
+            descriptions = d.pop(key)
+            if not isinstance(descriptions, list):
+                descriptions = [descriptions]
+            for description in descriptions:
+                # We want to make the identification of the measurement universal
+                for measurement in enums._MEASUREMENT_NAMESPACES:
+                    if measurement in description:
+                        name, item = measurement, description.pop(measurement)
+                        break
+                else:
+                    break
+                item['description'] = item.pop('item_description', None)
+                item['unit'] = item.pop('item_units', None)
+                if 'si_scale_code' in item:
+                    item['scale'] = item.pop('si_scale_code')
+                if 'pulse_factor' in item:
+                    item['pulse_factor'] = item.pop('pulse_factor')
+                description['measurement'] = {'name': name,
+                                              **item}
+            d[key + 's'] = descriptions
 
         # Promote the contents of the Qualified Event ID
         elif key == "qualified_event_id" and isinstance(d['qualified_event_id'], dict):
@@ -188,10 +190,6 @@ def normalize_dict(ordered_dict):
             d['event_id'] = qeid['event_id']
             d['modification_number'] = qeid['modification_number']
 
-        # Promote the contents of the tolerance items
-        # if key == "tolerance" and "tolerate" in d["tolerance"] and len(d["tolerance"]["tolerate"]) == 1:
-        #     d["tolerance"] = d["tolerance"]["tolerate"].values()[0]
-
         # Durations are encapsulated in their own object, remove this nesting
         elif isinstance(d[key], dict) and "duration" in d[key] and len(d[key]) == 1:
             d[key] = d[key]["duration"]
@@ -207,15 +205,26 @@ def normalize_dict(ordered_dict):
             else:
                 d[key] = [d[key][key[:-1]]]
 
-        # Payload values are wrapped in an object according to their type. We don't need that information.
+        # Payload values are wrapped in an object according to their type. We don't need that.
         elif key in ("signal_payload", "current_value"):
             value = d[key]
             if isinstance(d[key], dict):
-                if 'payload_float' in d[key] and 'value' in d[key]['payload_float'] and d[key]['payload_float']['value'] is not None:
+                if 'payload_float' in d[key] and 'value' in d[key]['payload_float'] \
+                        and d[key]['payload_float']['value'] is not None:
                     d[key] = float(d[key]['payload_float']['value'])
-                elif 'payload_int' in d[key] and 'value' in d[key]['payload_int'] and d[key]['payload_int'] is not None:
+                elif 'payload_int' in d[key] and 'value' in d[key]['payload_int'] \
+                        and d[key]['payload_int'] is not None:
                     d[key] = int(d[key]['payload_int']['value'])
 
+        # Report payloads contain an r_id and a type-wrapped payload_float
+        elif key == 'report_payload':
+            if 'payload_float' in d[key] and 'value' in d[key]['payload_float']:
+                v = d[key].pop('payload_float')
+                d[key]['value'] = float(v['value'])
+            elif 'payload_int' in d[key] and 'value' in d[key]['payload_int']:
+                v = d[key].pop('payload_float')
+                d[key]['value'] = int(v['value'])
+
         # All values other than 'false' must be interpreted as True for testEvent (rule 006)
         elif key == 'test_event' and not isinstance(d[key], bool):
             d[key] = True
@@ -240,56 +249,49 @@ def normalize_dict(ordered_dict):
             d.pop(key)
     return d
 
+
 def parse_datetime(value):
     """
     Parse an ISO8601 datetime into a datetime.datetime object.
     """
     matches = re.match(r'(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})\.?(\d{1,6})?\d*Z', value)
     if matches:
-        year, month, day, hour, minute, second, microsecond = (int(value) for value in matches.groups())
-        return datetime(year, month, day, hour, minute, second, microsecond=microsecond, tzinfo=timezone.utc)
+        year, month, day, hour, minute, second = (int(value)for value in matches.groups()[:-1])
+        micro = matches.groups()[-1]
+        if micro is None:
+            micro = 0
+        else:
+            micro = int(micro + "0" * (6 - len(micro)))
+        return datetime(year, month, day, hour, minute, second, micro, tzinfo=timezone.utc)
     else:
-        print(f"{value} did not match format")
+        logger.warning(f"parse_datetime: {value} did not match format")
         return value
 
+
 def parse_duration(value):
     """
     Parse a RFC5545 duration.
     """
-    # TODO: implement the full regex: matches = re.match(r'(\+|\-)?P((\d+Y)?(\d+M)?(\d+D)?T?(\d+H)?(\d+M)?(\d+S)?)|(\d+W)', value)
     if isinstance(value, timedelta):
         return value
-    matches = re.match(r'P(\d+(?:D|W))?T(\d+H)?(\d+M)?(\d+S)?', value)
+    regex = r'(\+|\-)?P(?:(?:(\d+)Y)?(?:(\d+)M)?(?:(\d+)D)?T?(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)S)?)|(?:(\d+)W)'
+    matches = re.match(regex, value)
     if not matches:
-        return False
-    days = hours = minutes = seconds = 0
-    _days, _hours, _minutes, _seconds = matches.groups()
-    if _days:
-        if _days.endswith("D"):
-            days = int(_days[:-1])
-        elif _days.endswith("W"):
-            days = int(_days[:-1]) * 7
-    if _hours:
-        hours = int(_hours[:-1])
-    if _minutes:
-        minutes = int(_minutes[:-1])
-    if _seconds:
-        seconds = int(_seconds[:-1])
-    return timedelta(days=days, hours=hours, minutes=minutes, seconds=seconds)
-
-def parse_int(value):
-    matches = re.match(r'^[\d-]+$', value)
-    if not matches:
-        return False
-    else:
-        return int(value)
+        raise ValueError(f"The duration '{value}' did not match the requested format")
+    years, months, days, hours, minutes, seconds, weeks = (int(g) if g else 0 for g in matches.groups()[1:])
+    if years != 0:
+        logger.warning("Received a duration that specifies years, which is not a determinate duration. "
+                       "It will be interpreted as 1 year = 365 days.")
+        days = days + 365 * years
+    if months != 0:
+        logger.warning("Received a duration that specifies months, which is not a determinate duration "
+                       "It will be interpreted as 1 month = 30 days.")
+        days = days + 30 * months
+    duration = timedelta(weeks=weeks, days=days, hours=hours, minutes=minutes, seconds=seconds)
+    if matches.groups()[0] == "-":
+        duration = -1 * duration
+    return duration
 
-def parse_float(value):
-    matches = re.match(r'^[\d.-]+$', value)
-    if not matches:
-        return False
-    else:
-        return float(value)
 
 def parse_boolean(value):
     if value == 'true':
@@ -297,16 +299,6 @@ def parse_boolean(value):
     else:
         return False
 
-def peek(iterable):
-    """
-    Peek into an iterable.
-    """
-    try:
-        first = next(iterable)
-    except StopIteration:
-        return None
-    else:
-        return itertools.chain([first], iterable)
 
 def datetimeformat(value, format=DATETIME_FORMAT):
     """
@@ -316,6 +308,7 @@ def datetimeformat(value, format=DATETIME_FORMAT):
         return value
     return value.astimezone(timezone.utc).strftime(format)
 
+
 def timedeltaformat(value):
     """
     Format a timedelta to a RFC5545 Duration.
@@ -329,7 +322,7 @@ def timedeltaformat(value):
     if days:
         formatted += f"{days}D"
     if hours or minutes or seconds:
-        formatted += f"T"
+        formatted += "T"
     if hours:
         formatted += f"{hours}H"
     if minutes:
@@ -338,24 +331,28 @@ def timedeltaformat(value):
         formatted += f"{seconds}S"
     return formatted
 
+
 def booleanformat(value):
     """
     Format a boolean value
     """
     if isinstance(value, bool):
-        if value == True:
+        if value is True:
             return "true"
-        elif value == False:
+        elif value is False:
             return "false"
     elif value in ("true", "false"):
         return value
     else:
         raise ValueError(f"A boolean value must be provided, not {value}.")
 
+
 def ensure_bytes(obj):
     """
     Converts a utf-8 str object to bytes.
     """
+    if obj is None:
+        return obj
     if isinstance(obj, bytes):
         return obj
     if isinstance(obj, str):
@@ -363,10 +360,13 @@ def ensure_bytes(obj):
     else:
         raise TypeError("Must be bytes or str")
 
+
 def ensure_str(obj):
     """
     Converts bytes to a utf-8 string.
     """
+    if obj is None:
+        return None
     if isinstance(obj, str):
         return obj
     if isinstance(obj, bytes):
@@ -374,13 +374,19 @@ def ensure_str(obj):
     else:
         raise TypeError("Must be bytes or str")
 
+
+def certificate_fingerprint_from_der(der_bytes):
+    hash = hashlib.sha256(der_bytes).digest().hex()
+    return ":".join([hash[i-2:i].upper() for i in range(-20, 0, 2)])
+
+
 def certificate_fingerprint(certificate_str):
     """
     Calculate the fingerprint for the given certificate, as defined by OpenADR.
     """
-    der_cert = ssl.PEM_cert_to_DER_cert(ensure_str(certificate_str))
-    hash = hashlib.sha256(der_cert).digest().hex()
-    return ":".join([hash[i-2:i].upper() for i in range(-20, 0, 2)])
+    der_bytes = ssl.PEM_cert_to_DER_cert(ensure_str(certificate_str))
+    return certificate_fingerprint_from_der(der_bytes)
+
 
 def extract_pem_cert(tree):
     """
@@ -393,3 +399,404 @@ def extract_pem_cert(tree):
     """
     cert = tree.find('.//{http://www.w3.org/2000/09/xmldsig#}X509Certificate').text
     return "-----BEGIN CERTIFICATE-----\n" + cert + "-----END CERTIFICATE-----\n"
+
+
+def find_by(dict_or_list, key, value, *args):
+    """
+    Find a dict inside a dict or list by key, value properties.
+    """
+    search_params = [(key, value)]
+    if args:
+        search_params += [(args[i], args[i+1]) for i in range(0, len(args), 2)]
+    if isinstance(dict_or_list, dict):
+        dict_or_list = dict_or_list.values()
+    for item in dict_or_list:
+        if not isinstance(item, dict):
+            _item = item.__dict__
+        else:
+            _item = item
+        for key, value in search_params:
+            if isinstance(value, tuple):
+                if key not in _item or _item[key] not in value:
+                    break
+            else:
+                if key not in _item or _item[key] != value:
+                    break
+        else:
+            return item
+    else:
+        return None
+
+
+def group_by(list_, key, pop_key=False):
+    """
+    Return a dict that groups values
+    """
+    grouped = {}
+    key_path = key.split(".")
+    for item in list_:
+        value = item
+        for key in key_path:
+            value = value.get(key)
+        if value not in grouped:
+            grouped[value] = []
+        grouped[value].append(item)
+    return grouped
+
+
+def cron_config(interval, randomize_seconds=False):
+    """
+    Returns a dict with cron settings for the given interval
+    """
+    if interval < timedelta(minutes=1):
+        second = f"*/{interval.seconds}"
+        minute = "*"
+        hour = "*"
+    elif interval < timedelta(hours=1):
+        second = "0"
+        minute = f"*/{int(interval.total_seconds()/60)}"
+        hour = "*"
+    elif interval < timedelta(hours=24):
+        second = "0"
+        minute = "0"
+        hour = f"*/{int(interval.total_seconds()/3600)}"
+    else:
+        second = "0"
+        minute = "0"
+        hour = "0"
+    cron_config = {"second": second, "minute": minute, "hour": hour}
+    if randomize_seconds:
+        jitter = min(int(interval.total_seconds() / 10), 300)
+        cron_config['jitter'] = jitter
+    return cron_config
+
+
+def get_cert_fingerprint_from_request(request):
+    ssl_object = request.transport.get_extra_info('ssl_object')
+    if ssl_object:
+        der_bytes = ssl_object.getpeercert(binary_form=True)
+        if der_bytes:
+            return certificate_fingerprint_from_der(der_bytes)
+
+
+def group_targets_by_type(list_of_targets):
+    targets_by_type = {}
+    for target in list_of_targets:
+        for key, value in target.items():
+            if value is None:
+                continue
+            if key not in targets_by_type:
+                targets_by_type[key] = []
+            targets_by_type[key].append(value)
+    return targets_by_type
+
+
+def ungroup_targets_by_type(targets_by_type):
+    ungrouped_targets = []
+    for target_type, targets in targets_by_type.items():
+        if isinstance(targets, list):
+            for target in targets:
+                ungrouped_targets.append({target_type: target})
+        elif isinstance(targets, str):
+            ungrouped_targets.append({target_type: targets})
+    return ungrouped_targets
+
+
+def validate_report_measurement_dict(measurement):
+    from openleadr.enums import _ACCEPTABLE_UNITS, _MEASUREMENT_DESCRIPTIONS
+
+    if 'name' not in measurement \
+            or 'description' not in measurement \
+            or 'unit' not in measurement:
+        raise ValueError("The measurement dict must contain the following keys: "
+                         "'name', 'description', 'unit'. Please correct this.")
+
+    name = measurement['name']
+    description = measurement['description']
+    unit = measurement['unit']
+
+    # Validate the item name and description match
+    if name in _MEASUREMENT_DESCRIPTIONS:
+        required_description = _MEASUREMENT_DESCRIPTIONS[name]
+        if description != required_description:
+            if description.lower() == required_description.lower():
+                logger.warning(f"The description for the measurement with name '{name}' "
+                               f"was not in the correct case; you provided '{description}' but "
+                               f"it should be '{required_description}'. "
+                               "This was automatically corrected.")
+                measurement['description'] = required_description
+            else:
+                raise ValueError(f"The measurement's description '{description}' "
+                                 f"did not match the expected description for this type "
+                                 f" ('{required_description}'). Please correct this, or use "
+                                 "'customUnit' as the name.")
+        if unit not in _ACCEPTABLE_UNITS[name]:
+            raise ValueError(f"The unit '{unit}' is not acceptable for measurement '{name}'. Allowed "
+                             f"units are: '" + "', '".join(_ACCEPTABLE_UNITS[name]) + "'.")
+    else:
+        if name != 'customUnit':
+            logger.warning(f"You provided a measurement with an unknown name {name}. "
+                           "This was corrected to 'customUnit'. Please correct this in your "
+                           "report definition.")
+            measurement['name'] = 'customUnit'
+
+    if 'power' in name:
+        if 'power_attributes' in measurement:
+            power_attributes = measurement['power_attributes']
+            if 'voltage' not in power_attributes \
+                    or 'ac' not in power_attributes \
+                    or 'hertz' not in power_attributes:
+                raise ValueError("The power_attributes of the measurement must contain the "
+                                 "following keys: 'voltage' (int), 'ac' (bool), 'hertz' (int).")
+        else:
+            raise ValueError("A 'power' related measurement must contain a "
+                             "'power_attributes' section that contains the following "
+                             "keys: 'voltage' (int), 'ac' (boolean), 'hertz' (int)")
+
+
+def get_active_period_from_intervals(intervals, as_dict=True):
+    if is_dataclass(intervals[0]):
+        intervals = [asdict(i) for i in intervals]
+    period_start = min([i['dtstart'] for i in intervals])
+    period_duration = max([i['dtstart'] + i['duration'] - period_start for i in intervals])
+    if as_dict:
+        return {'dtstart': period_start,
+                'duration': period_duration}
+    else:
+        from openleadr.objects import ActivePeriod
+        return ActivePeriod(dtstart=period_start, duration=period_duration)
+
+
+def determine_event_status(active_period):
+    now = datetime.now(timezone.utc)
+    active_period_start = getmember(active_period, 'dtstart')
+    if active_period_start.tzinfo is None:
+        active_period_start = active_period_start.astimezone(timezone.utc)
+        setmember(active_period, 'dtstart', active_period_start)
+    active_period_end = active_period_start + getmember(active_period, 'duration')
+    if now >= active_period_end:
+        return 'completed'
+    if now >= active_period_start:
+        return 'active'
+    if getmember(active_period, 'ramp_up_period', None) is not None:
+        ramp_up_start = active_period_start - getmember(active_period, 'ramp_up_period')
+        if now >= ramp_up_start:
+            return 'near'
+    return 'far'
+
+
+async def delayed_call(func, delay):
+    try:
+        if isinstance(delay, timedelta):
+            delay = delay.total_seconds()
+        await asyncio.sleep(delay)
+        if asyncio.iscoroutinefunction(func):
+            await func()
+        elif asyncio.iscoroutine(func):
+            await func
+        else:
+            func()
+    except asyncio.CancelledError:
+        pass
+
+
+def hasmember(obj, member):
+    """
+    Check if a dict or dataclass has the given member
+    """
+    if is_dataclass(obj):
+        if hasattr(obj, member):
+            return True
+    else:
+        if member in obj:
+            return True
+    return False
+
+
+def getmember(obj, member, missing='_RAISE_'):
+    """
+    Get a member from a dict or dataclass
+    """
+    if is_dataclass(obj):
+        if not missing == '_RAISE_' and not hasattr(obj, member):
+            return missing
+        else:
+            return getattr(obj, member)
+    else:
+        if missing == '_RAISE_':
+            return obj[member]
+        else:
+            return obj.get(member, missing)
+
+
+def setmember(obj, member, value):
+    """
+    Set a member of a dict of dataclass
+    """
+    if is_dataclass(obj):
+        setattr(obj, member, value)
+    else:
+        obj[member] = value
+
+
+def get_next_event_from_deque(deque):
+    unused_elements = []
+    event = None
+    for i in range(len(deque)):
+        msg = deque.popleft()
+        if isinstance(msg, objects.Event) or (isinstance(msg, dict) and 'event_descriptor' in msg):
+            event = msg
+            break
+        else:
+            unused_elements.append(msg)
+    deque.extend(unused_elements)
+    return event
+
+
+def validate_report_request_tuples(list_of_report_requests, mode='full'):
+    if len(list_of_report_requests) == 0:
+        return
+    for report_requests in list_of_report_requests:
+        if report_requests is None:
+            continue
+        for i, rrq in enumerate(report_requests):
+            if rrq is None:
+                continue
+
+            # Check if it is a tuple
+            elif not isinstance(rrq, tuple):
+                report_requests[i] = None
+                if mode == 'full':
+                    logger.error("Your on_register_report handler did not return a list of tuples. "
+                                 f"The first item from the list was '{rrq}' ({rrq.__class__.__name__}).")
+                else:
+                    logger.error("Your on_register_report handler did not return a tuple. "
+                                 f"It returned '{rrq}'. Please see the documentation for the correct format.")
+
+            # Check if it has the correct length
+            elif not len(rrq) in (3, 4):
+                report_requests[i] = None
+                if mode == 'full':
+                    logger.error("Your on_register_report handler returned tuples of the wrong length. "
+                                 f"It should be 3 or 4. It returned: '{rrq}'.")
+                else:
+                    logger.error("Your on_register_report handler returned a tuple of the wrong length. "
+                                 f"It should be 2 or 3. It returned: '{rrq[1:]}'.")
+
+            # Check if the first element is callable
+            elif not callable(rrq[1]):
+                report_requests[i] = None
+                if mode == 'full':
+                    logger.error(f"Your on_register_report handler did not return the correct tuple. "
+                                 "It should return a list of (r_id, callback, sampling_interval) or "
+                                 "(r_id, callback, sampling_interval, reporting_interval) tuples, where "
+                                 "the r_id is a string, callback is a callable function or coroutine, and "
+                                 "sampling_interval and reporting_interval are of type datetime.timedelta. "
+                                 f"It returned: '{rrq}'. The second element was not callable.")
+                else:
+                    logger.error(f"Your on_register_report handler did not return the correct tuple. "
+                                 "It should return a (callback, sampling_interval) or "
+                                 "(callback, sampling_interval, reporting_interval) tuple, where "
+                                 "the callback is a callable function or coroutine, and "
+                                 "sampling_interval and reporting_interval are of type datetime.timedelta. "
+                                 f"It returned: '{rrq[1:]}'. The first element was not callable.")
+
+            # Check if the second element is a timedelta
+            elif not isinstance(rrq[2], timedelta):
+                report_requests[i] = None
+                if mode == 'full':
+                    logger.error(f"Your on_register_report handler did not return the correct tuple. "
+                                 "It should return a list of (r_id, callback, sampling_interval) or "
+                                 "(r_id, callback, sampling_interval, reporting_interval) tuples, where "
+                                 "sampling_interval and reporting_interval are of type datetime.timedelta. "
+                                 f"It returned: '{rrq}'. The third element was not of type timedelta.")
+                else:
+                    logger.error(f"Your on_register_report handler did not return the correct tuple. "
+                                 "It should return a (callback, sampling_interval) or "
+                                 "(callback, sampling_interval, reporting_interval) tuple, where "
+                                 "sampling_interval and reporting_interval are of type datetime.timedelta. "
+                                 f"It returned: '{rrq[1:]}'. The second element was not of type timedelta.")
+
+            # Check if the third element is a timedelta (if it exists)
+            elif len(rrq) == 4 and not isinstance(rrq[3], timedelta):
+                report_requests[i] = None
+                if mode == 'full':
+                    logger.error(f"Your on_register_report handler did not return the correct tuple. "
+                                 "It should return a list of (r_id, callback, sampling_interval) or "
+                                 "(r_id, callback, sampling_interval, reporting_interval) tuples, where "
+                                 "sampling_interval and reporting_interval are of type datetime.timedelta. "
+                                 f"It returned: '{rrq}'. The fourth element was not of type timedelta.")
+                else:
+                    logger.error(f"Your on_register_report handler did not return the correct tuple. "
+                                 "It should return a (callback, sampling_interval) or "
+                                 "(callback, sampling_interval, reporting_interval) tuple, where "
+                                 "sampling_interval and reporting_interval are of type datetime.timedelta. "
+                                 f"It returned: '{rrq[1:]}'. The third element was not of type timedelta.")
+
+
+async def await_if_required(result):
+    if asyncio.iscoroutine(result):
+        result = await result
+    return result
+
+
+async def gather_if_required(results):
+    if results is None:
+        return results
+    if len(results) > 0:
+        if not any([asyncio.iscoroutine(r) for r in results]):
+            results = results
+        elif all([asyncio.iscoroutine(r) for r in results]):
+            results = await asyncio.gather(*results)
+        else:
+            results = [await await_if_required(result) for result in results]
+    return results
+
+
+def order_events(events, limit=None, offset=None):
+    """
+    Order the events according to the OpenADR rules:
+    - active events before inactive events
+    - high priority before low priority
+    - earlier before later
+    """
+    def event_priority(event):
+        # The default and lowest priority is 0, which we should interpret as a high value.
+        priority = getmember(getmember(event, 'event_descriptor'), 'priority', float('inf'))
+        if priority == 0:
+            priority = float('inf')
+        return priority
+
+    if events is None:
+        return None
+    if isinstance(events, objects.Event):
+        events = [events]
+    elif isinstance(events, dict):
+        events = [events]
+
+    # Update the event statuses
+    for event in events:
+        event_status = determine_event_status(getmember(event, 'active_period'))
+        setmember(getmember(event, 'event_descriptor'), 'event_status', event_status)
+
+    # Short circuit if we only have one event:
+    if len(events) == 1:
+        return events
+
+    # Get all the active events first
+    active_events = [event for event in events if getmember(getmember(event, 'event_descriptor'), 'event_status') == 'active']
+    other_events = [event for event in events if getmember(getmember(event, 'event_descriptor'), 'event_status') != 'active']
+
+    # Sort the active events by priority
+    active_events.sort(key=lambda e: event_priority(e))
+
+    # Sort the active events by start date
+    active_events.sort(key=lambda e: getmember(getmember(e, 'active_period'), 'dtstart'))
+
+    # Sort the non-active events by their start date
+    other_events.sort(key=lambda e: getmember(getmember(e, 'active_period'), 'dtstart'))
+
+    ordered_events = active_events + other_events
+    if limit and offset:
+        return ordered_events[offset:offset+limit]
+    return ordered_events

schema/xmldsig-core-schema.xsd

@@ -1,318 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<!DOCTYPE schema
-  PUBLIC "-//W3C//DTD XMLSchema 200102//EN" "http://www.w3.org/2001/XMLSchema.dtd"
- [
-   <!ATTLIST schema 
-     xmlns:ds CDATA #FIXED "http://www.w3.org/2000/09/xmldsig#">
-   <!ENTITY dsig 'http://www.w3.org/2000/09/xmldsig#'>
-   <!ENTITY % p ''>
-   <!ENTITY % s ''>
-  ]>
-
-<!-- Schema for XML Signatures
-    http://www.w3.org/2000/09/xmldsig#
-    $Revision: 1.1 $ on $Date: 2002/02/08 20:32:26 $ by $Author: reagle $
-
-    Copyright 2001 The Internet Society and W3C (Massachusetts Institute
-    of Technology, Institut National de Recherche en Informatique et en
-    Automatique, Keio University). All Rights Reserved.
-    http://www.w3.org/Consortium/Legal/
-
-    This document is governed by the W3C Software License [1] as described
-    in the FAQ [2].
-
-    [1] http://www.w3.org/Consortium/Legal/copyright-software-19980720
-    [2] http://www.w3.org/Consortium/Legal/IPR-FAQ-20000620.html#DTD
--->
-
-
-<schema xmlns="http://www.w3.org/2001/XMLSchema"
-        xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
-        targetNamespace="http://www.w3.org/2000/09/xmldsig#"
-        version="0.1" elementFormDefault="qualified"> 
-
-<!-- Basic Types Defined for Signatures -->
-
-<simpleType name="CryptoBinary">
-  <restriction base="base64Binary">
-  </restriction>
-</simpleType>
-
-<!-- Start Signature -->
-
-<element name="Signature" type="ds:SignatureType"/>
-<complexType name="SignatureType">
-  <sequence> 
-    <element ref="ds:SignedInfo"/> 
-    <element ref="ds:SignatureValue"/> 
-    <element ref="ds:KeyInfo" minOccurs="0"/> 
-    <element ref="ds:Object" minOccurs="0" maxOccurs="unbounded"/> 
-  </sequence>  
-  <attribute name="Id" type="ID" use="optional"/>
-</complexType>
-
-  <element name="SignatureValue" type="ds:SignatureValueType"/> 
-  <complexType name="SignatureValueType">
-    <simpleContent>
-      <extension base="base64Binary">
-        <attribute name="Id" type="ID" use="optional"/>
-      </extension>
-    </simpleContent>
-  </complexType>
-
-<!-- Start SignedInfo -->
-
-<element name="SignedInfo" type="ds:SignedInfoType"/>
-<complexType name="SignedInfoType">
-  <sequence> 
-    <element ref="ds:CanonicalizationMethod"/> 
-    <element ref="ds:SignatureMethod"/> 
-    <element ref="ds:Reference" maxOccurs="unbounded"/> 
-  </sequence>  
-  <attribute name="Id" type="ID" use="optional"/> 
-</complexType>
-
-  <element name="CanonicalizationMethod" type="ds:CanonicalizationMethodType"/> 
-  <complexType name="CanonicalizationMethodType" mixed="true">
-    <sequence>
-      <any namespace="##any" minOccurs="0" maxOccurs="unbounded"/>
-      <!-- (0,unbounded) elements from (1,1) namespace -->
-    </sequence>
-    <attribute name="Algorithm" type="anyURI" use="required"/> 
-  </complexType>
-
-  <element name="SignatureMethod" type="ds:SignatureMethodType"/>
-  <complexType name="SignatureMethodType" mixed="true">
-    <sequence>
-      <element name="HMACOutputLength" minOccurs="0" type="ds:HMACOutputLengthType"/>
-      <any namespace="##other" minOccurs="0" maxOccurs="unbounded"/>
-      <!-- (0,unbounded) elements from (1,1) external namespace -->
-    </sequence>
-    <attribute name="Algorithm" type="anyURI" use="required"/> 
-  </complexType>
-
-<!-- Start Reference -->
-
-<element name="Reference" type="ds:ReferenceType"/>
-<complexType name="ReferenceType">
-  <sequence> 
-    <element ref="ds:Transforms" minOccurs="0"/> 
-    <element ref="ds:DigestMethod"/> 
-    <element ref="ds:DigestValue"/> 
-  </sequence>
-  <attribute name="Id" type="ID" use="optional"/> 
-  <attribute name="URI" type="anyURI" use="optional"/> 
-  <attribute name="Type" type="anyURI" use="optional"/> 
-</complexType>
-
-  <element name="Transforms" type="ds:TransformsType"/>
-  <complexType name="TransformsType">
-    <sequence>
-      <element ref="ds:Transform" maxOccurs="unbounded"/>  
-    </sequence>
-  </complexType>
-
-  <element name="Transform" type="ds:TransformType"/>
-  <complexType name="TransformType" mixed="true">
-    <choice minOccurs="0" maxOccurs="unbounded"> 
-      <any namespace="##other" processContents="lax"/>
-      <!-- (1,1) elements from (0,unbounded) namespaces -->
-      <element name="XPath" type="string"/> 
-    </choice>
-    <attribute name="Algorithm" type="anyURI" use="required"/> 
-  </complexType>
-
-<!-- End Reference -->
-
-<element name="DigestMethod" type="ds:DigestMethodType"/>
-<complexType name="DigestMethodType" mixed="true"> 
-  <sequence>
-    <any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>
-  </sequence>    
-  <attribute name="Algorithm" type="anyURI" use="required"/> 
-</complexType>
-
-<element name="DigestValue" type="ds:DigestValueType"/>
-<simpleType name="DigestValueType">
-  <restriction base="base64Binary"/>
-</simpleType>
-
-<!-- End SignedInfo -->
-
-<!-- Start KeyInfo -->
-
-<element name="KeyInfo" type="ds:KeyInfoType"/> 
-<complexType name="KeyInfoType" mixed="true">
-  <choice maxOccurs="unbounded">     
-    <element ref="ds:KeyName"/> 
-    <element ref="ds:KeyValue"/> 
-    <element ref="ds:RetrievalMethod"/> 
-    <element ref="ds:X509Data"/> 
-    <element ref="ds:PGPData"/> 
-    <element ref="ds:SPKIData"/>
-    <element ref="ds:MgmtData"/>
-    <any processContents="lax" namespace="##other"/>
-    <!-- (1,1) elements from (0,unbounded) namespaces -->
-  </choice>
-  <attribute name="Id" type="ID" use="optional"/> 
-</complexType>
-
-  <element name="KeyName" type="string"/>
-  <element name="MgmtData" type="string"/>
-
-  <element name="KeyValue" type="ds:KeyValueType"/> 
-  <complexType name="KeyValueType" mixed="true">
-   <choice>
-     <element ref="ds:DSAKeyValue"/>
-     <element ref="ds:RSAKeyValue"/>
-     <any namespace="##other" processContents="lax"/>
-   </choice>
-  </complexType>
-
-  <element name="RetrievalMethod" type="ds:RetrievalMethodType"/> 
-  <complexType name="RetrievalMethodType">
-    <sequence>
-      <element ref="ds:Transforms" minOccurs="0"/> 
-    </sequence>  
-    <attribute name="URI" type="anyURI"/>
-    <attribute name="Type" type="anyURI" use="optional"/>
-  </complexType>
-
-<!-- Start X509Data -->
-
-<element name="X509Data" type="ds:X509DataType"/> 
-<complexType name="X509DataType">
-  <sequence maxOccurs="unbounded">
-    <choice>
-      <element name="X509IssuerSerial" type="ds:X509IssuerSerialType"/>
-      <element name="X509SKI" type="base64Binary"/>
-      <element name="X509SubjectName" type="string"/>
-      <element name="X509Certificate" type="base64Binary"/>
-      <element name="X509CRL" type="base64Binary"/>
-      <any namespace="##other" processContents="lax"/>
-    </choice>
-  </sequence>
-</complexType>
-
-<complexType name="X509IssuerSerialType"> 
-  <sequence> 
-    <element name="X509IssuerName" type="string"/> 
-    <element name="X509SerialNumber" type="integer"/> 
-  </sequence>
-</complexType>
-
-<!-- End X509Data -->
-
-<!-- Begin PGPData -->
-
-<element name="PGPData" type="ds:PGPDataType"/> 
-<complexType name="PGPDataType"> 
-  <choice>
-    <sequence>
-      <element name="PGPKeyID" type="base64Binary"/> 
-      <element name="PGPKeyPacket" type="base64Binary" minOccurs="0"/> 
-      <any namespace="##other" processContents="lax" minOccurs="0"
-       maxOccurs="unbounded"/>
-    </sequence>
-    <sequence>
-      <element name="PGPKeyPacket" type="base64Binary"/> 
-      <any namespace="##other" processContents="lax" minOccurs="0"
-       maxOccurs="unbounded"/>
-    </sequence>
-  </choice>
-</complexType>
-
-<!-- End PGPData -->
-
-<!-- Begin SPKIData -->
-
-<element name="SPKIData" type="ds:SPKIDataType"/> 
-<complexType name="SPKIDataType">
-  <sequence maxOccurs="unbounded">
-    <element name="SPKISexp" type="base64Binary"/>
-    <any namespace="##other" processContents="lax" minOccurs="0"/>
-  </sequence>
-</complexType> 
-
-<!-- End SPKIData -->
-
-<!-- End KeyInfo -->
-
-<!-- Start Object (Manifest, SignatureProperty) -->
-
-<element name="Object" type="ds:ObjectType"/> 
-<complexType name="ObjectType" mixed="true">
-  <sequence minOccurs="0" maxOccurs="unbounded">
-    <any namespace="##any" processContents="lax"/>
-  </sequence>
-  <attribute name="Id" type="ID" use="optional"/> 
-  <attribute name="MimeType" type="string" use="optional"/> <!-- add a grep facet -->
-  <attribute name="Encoding" type="anyURI" use="optional"/> 
-</complexType>
-
-<element name="Manifest" type="ds:ManifestType"/> 
-<complexType name="ManifestType">
-  <sequence>
-    <element ref="ds:Reference" maxOccurs="unbounded"/> 
-  </sequence>
-  <attribute name="Id" type="ID" use="optional"/> 
-</complexType>
-
-<element name="SignatureProperties" type="ds:SignaturePropertiesType"/> 
-<complexType name="SignaturePropertiesType">
-  <sequence>
-    <element ref="ds:SignatureProperty" maxOccurs="unbounded"/> 
-  </sequence>
-  <attribute name="Id" type="ID" use="optional"/> 
-</complexType>
-
-   <element name="SignatureProperty" type="ds:SignaturePropertyType"/> 
-   <complexType name="SignaturePropertyType" mixed="true">
-     <choice maxOccurs="unbounded">
-       <any namespace="##other" processContents="lax"/>
-       <!-- (1,1) elements from (1,unbounded) namespaces -->
-     </choice>
-     <attribute name="Target" type="anyURI" use="required"/> 
-     <attribute name="Id" type="ID" use="optional"/> 
-   </complexType>
-
-<!-- End Object (Manifest, SignatureProperty) -->
-
-<!-- Start Algorithm Parameters -->
-
-<simpleType name="HMACOutputLengthType">
-  <restriction base="integer"/>
-</simpleType>
-
-<!-- Start KeyValue Element-types -->
-
-<element name="DSAKeyValue" type="ds:DSAKeyValueType"/>
-<complexType name="DSAKeyValueType">
-  <sequence>
-    <sequence minOccurs="0">
-      <element name="P" type="ds:CryptoBinary"/>
-      <element name="Q" type="ds:CryptoBinary"/>
-    </sequence>
-    <element name="G" type="ds:CryptoBinary" minOccurs="0"/>
-    <element name="Y" type="ds:CryptoBinary"/>
-    <element name="J" type="ds:CryptoBinary" minOccurs="0"/>
-    <sequence minOccurs="0">
-      <element name="Seed" type="ds:CryptoBinary"/>
-      <element name="PgenCounter" type="ds:CryptoBinary"/>
-    </sequence>
-  </sequence>
-</complexType>
-
-<element name="RSAKeyValue" type="ds:RSAKeyValueType"/>
-<complexType name="RSAKeyValueType">
-  <sequence>
-    <element name="Modulus" type="ds:CryptoBinary"/> 
-    <element name="Exponent" type="ds:CryptoBinary"/> 
-  </sequence>
-</complexType> 
-
-<!-- End KeyValue Element-types -->
-
-<!-- End Signature -->
-
-</schema>

setup.py

@@ -15,20 +15,20 @@
 # limitations under the License.
 
 from setuptools import setup
-import os
 
 with open('README.md', 'r') as fh:
     long_description = fh.read()
 
-with open(os.path.join(os.path.dirname(os.path.abspath(__file__)), 'VERSION')) as file:
-    VERSION = file.read().strip()
-
 setup(name='openleadr',
-      version=VERSION,
-      description='Python library for dealing with OpenADR',
+      version='0.5.18',
+      description='Python3 library for building OpenADR Clients (VENs) and Servers (VTNs)',
       long_description=long_description,
       long_description_content_type='text/markdown',
-      url='https://openleadr.elaad.io',
+      url='https://openleadr.org',
+      project_urls={'GitHub': 'https://github.com/openleadr/openleadr-python',
+                    'Documentation': 'https://openleadr.org/docs'},
       packages=['openleadr', 'openleadr.service'],
+      python_requires='>=3.7.0',
       include_package_data=True,
-      install_requires=['xmltodict', 'aiohttp', 'apscheduler', 'jinja2', 'signxml-openadr>2.8.0'])
+      install_requires=['xmltodict', 'aiohttp', 'apscheduler', 'jinja2', 'signxml-openadr==2.9.1'],
+      entry_points={'console_scripts': ['fingerprint = openleadr.fingerprint:show_fingerprint']})

test/cert.pem

@@ -1,32 +0,0 @@
------BEGIN CERTIFICATE-----
-MIIFiTCCA3GgAwIBAgIUIIpr283UlljB8eqbykYRd/j75h4wDQYJKoZIhvcNAQEL
-BQAwVDELMAkGA1UEBhMCTkwxEzARBgNVBAgMCkdlbGRlcmxhbmQxDzANBgNVBAcM
-BkFybmhlbTEfMB0GA1UECgwWT3BlbkxFQURSIE9yZ2FuaXphdGlvbjAeFw0yMDA5
-MDkwNzU3MTRaFw0zMDA5MDcwNzU3MTRaMFQxCzAJBgNVBAYTAk5MMRMwEQYDVQQI
-DApHZWxkZXJsYW5kMQ8wDQYDVQQHDAZBcm5oZW0xHzAdBgNVBAoMFk9wZW5MRUFE
-UiBPcmdhbml6YXRpb24wggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDM
-pEJj7sovh0EAQdCJW85cSPJ0UQZ137wGAP05sVjgh8k9kfyIbrsnUnhvByI/NHAQ
-nyXtJP8rvvakgsNj/YizzgYO8n6s69NGi0mnZCTV9gjUlt2HZ8v/UTkL4otEhXSw
-3r/B2vrTqfhNkJI/b9cJAaQGwLLc5TCC+NzukXUl4BtBv5js1Z29mnvsGHgvxzoA
-jcWBY52KHvAZvt0GsShyOje2E71gch/tMfKipqlNB2Cbmq9gFnGyJTJncbHoGMxr
-7is5v5bzJOXuJj/Eztbcj2rH31ltV1jBbAdWfcKGbMmTPNZTOvsDO6L2M0U5fiFR
-24fSfACd9IvvbiBgDIo9dMxdC2hJ3MwqJKK6L03ungDiXJyCtQqwgYtlSpfjI0tz
-kPs29pYtBqAdQEMOrdMcNn+94O6Axylr1fjTg3d41w6X7IOhfKxyG285fk6ad4DH
-3RrRRdjsO6LZUgSKzpxKGlGpcRKMgpMI360L44NJEh598W2whRjcozvNicqlWGze
-eu+zEeeFIta+9vfFxpI0aLLBUq3JZm2cdKSBfHGyBBV9ids0CfeIBFrwB0vVsF22
-UHg3t7LTfLaP4xlrRvlITzy2l0eJhDNPIWSXXA9Hl2+XQoYf4iYwY628loXPeyAd
-nXmMse0KbSJtUiuDqQOLuyF8aty4hNLzJ0K5w+OPyQIDAQABo1MwUTAdBgNVHQ4E
-FgQUjHzJIuGIAx5lr1bVjefd1Xi1/tYwHwYDVR0jBBgwFoAUjHzJIuGIAx5lr1bV
-jefd1Xi1/tYwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAgEAcCGc
-jjqT1LaJijJAY/EQxA/baqS3/d9dTH1A7khPTm9dtFAg1Nlrovhul4pi+yK9F4np
-iGgHsYx2IafuvTZ4DlcimtMfNRuH11BZsVQoBqJH3QPDa2s99zUnGN0XRYDgsh50
-9li9YW1PsNfcVX0YTkOMUgF9vEqiSnvpNnfdINEA/MJQysrERkfdzDO8pETQ0/cb
-VPpYStu4mJ1MQMQKuieYGtOkvzFWFT1aeQg1UMUATfBbWCC7vkbupMzhLiYFh+ZT
-ny17vhUMFMAr7Kt6cil+CavK9x88NRk4i1Jop1c/Nz6OlB67q4/Zhiz7NUKAUp0v
-iyWnelpg/jQ2tfFp6ThaIGX4Jgz1oa/w6QCCUCSwqZeuXl1sLZ8NyNsssGPLUuDt
-mbfmpVMoBleQB/J94P6QJtr9yCBWokUNrZGugmnrGuLR/FSBmq9C/hiuVnwM1uCM
-8yPPkjaD2pewXDzarrg9dEFd1G4WSv+YLj9uhuVILnS1GKzHBJtslYL7aVXvMlcx
-ER4rrcWunQBZAyWg8FqA+RlX30mDiHIDsZBl2/1t1yKCifhnMIgEx0kKk4KrD4CR
-f22dfslN4950FRhpA51rBdxKcJnw8XziucG8VGKnTONCF30IPDeT1eHj/th2m28S
-/Y2BqiYt4MxrrOmaEAaqupRYzH6FhG+jbgGIyVA=
------END CERTIFICATE-----

test/conformance/test_conformance_008.py

@@ -22,11 +22,11 @@ from openleadr.messaging import create_message, parse_message
 from datetime import datetime, timezone, timedelta
 
 from pprint import pprint
-import warnings
+import logging
 
 
 @pytest.mark.asyncio
-async def test_conformance_008_autocorrect():
+async def test_conformance_008_autocorrect(caplog):
     """
     oadrDistributeEvent eventSignal interval durations for a given event MUST
     add up to eiEvent eiActivePeriod duration.
@@ -60,14 +60,15 @@ async def test_conformance_008_autocorrect():
         }
 
     # Create a message with this event
-    with pytest.warns(UserWarning):
-        msg = create_message('oadrDistributeEvent',
-                             response={'response_code': 200,
-                                       'response_description': 'OK',
-                                       'request_id': generate_id()},
-                             request_id=generate_id(),
-                             vtn_id=generate_id(),
-                             events=[event])
+    msg = create_message('oadrDistributeEvent',
+                         response={'response_code': 200,
+                                   'response_description': 'OK',
+                                   'request_id': generate_id()},
+                         request_id=generate_id(),
+                         vtn_id=generate_id(),
+                         events=[event])
+
+    assert caplog.record_tuples == [("openleadr", logging.WARNING, f"The active_period duration for event {event_id} (0:05:00) differs from the sum of the interval's durations (0:30:00). The active_period duration has been adjusted to (0:30:00).")]
 
     parsed_type, parsed_msg = parse_message(msg)
     assert parsed_type == 'oadrDistributeEvent'

test/conformance/test_conformance_014.py

@@ -20,13 +20,11 @@ from openleadr import OpenADRClient, OpenADRServer, enums
 from openleadr.utils import generate_id
 from openleadr.messaging import create_message, parse_message
 from datetime import datetime, timezone, timedelta
-
+import logging
 from pprint import pprint
-import warnings
-
 
 @pytest.mark.asyncio
-async def test_conformance_014_warn():
+async def test_conformance_014_warn(caplog):
     """
     If currentValue is included in the payload, it MUST be set to 0 (normal)
     when the event status is not “active” for the SIMPLE signalName.
@@ -61,11 +59,13 @@ async def test_conformance_014_warn():
         }
 
     # Create a message with this event
-    with pytest.warns(UserWarning):
-        msg = create_message('oadrDistributeEvent',
-                             response={'response_code': 200,
-                                       'response_description': 'OK',
-                                       'request_id': generate_id()},
-                             request_id=generate_id(),
-                             vtn_id=generate_id(),
-                             events=[event])
+    msg = create_message('oadrDistributeEvent',
+                         response={'response_code': 200,
+                                   'response_description': 'OK',
+                                   'request_id': generate_id()},
+                         request_id=generate_id(),
+                         vtn_id=generate_id(),
+                         events=[event])
+
+    assert caplog.record_tuples == [("openleadr", logging.WARNING, "The current_value for a SIMPLE event that is not yet active must be 0. This will be corrected.")]
+

test/conformance/test_conformance_021.py

@@ -17,18 +17,98 @@
 import pytest
 
 from openleadr import OpenADRClient, OpenADRServer, enums
-from openleadr.utils import generate_id
+from openleadr.utils import generate_id, datetimeformat, timedeltaformat, booleanformat
 from openleadr.messaging import create_message, parse_message
 from openleadr.objects import Event, EventDescriptor, ActivePeriod, EventSignal, Interval
 from datetime import datetime, timezone, timedelta
 
+import json
+import sqlite3
 from pprint import pprint
 import warnings
 
-from test.fixtures.simple_server import start_server, add_event
+VEN_NAME = 'myven'
+VTN_ID = "TestVTN"
+
+async def lookup_ven(ven_name=None, ven_id=None):
+    """
+    Look up a ven by its name or ID
+    """
+    return {'ven_id': '1234'}
+
+async def on_update_report(report, futures=None):
+    if futures:
+        futures.pop().set_result(True)
+    pass
+
+async def on_register_report(report, futures=None):
+    """
+    Deal with this report.
+    """
+    if futures:
+        futures.pop().set_result(True)
+    granularity = min(*[rd['sampling_rate']['min_period'] for rd in report['report_descriptions']])
+    return (on_update_report, granularity, [rd['r_id'] for rd in report['report_descriptions']])
+
+async def on_create_party_registration(ven_name, future=None):
+    if future:
+        future.set_result(True)
+    ven_id = '1234'
+    registration_id = 'abcd'
+    return ven_id, registration_id
+
+class EventFormatter(json.JSONEncoder):
+    def default(self, obj):
+        if isinstance(obj, timedelta):
+            return timedeltaformat(obj)
+        if isinstance(obj, datetime):
+            return datetimeformat(obj)
+        if isinstance(obj, bool):
+            return booleanformat(obj)
+        return json.JSONEncoder.default(self, obj)
+
+DB = sqlite3.connect(":memory:")
+with DB:
+    DB.execute("CREATE TABLE vens (ven_id STRING, ven_name STRING, online BOOLEAN, last_seen DATETIME, registration_id STRING)")
+    DB.execute("CREATE TABLE events (event_id STRING, ven_id STRING, request_id STRING, status STRING, event JSON, created_at DATETIME, updated_at DATETIME)")
+
+def add_ven(ven_name, ven_id, registration_id):
+    with DB:
+        DB.execute("""INSERT INTO vens (ven_id, ven_name, online, last_seen, registration_id)
+                           VALUES (?, ?, ?, ?, ?)""", (ven_id, ven_name, True, datetime.now().replace(microsecond=0), registration_id))
+
+def add_event(ven_id, event_id, event):
+    serialized_event = json.dumps(event, cls=EventFormatter)
+    with DB:
+        DB.execute("""INSERT INTO events (ven_id, event_id, request_id, status, event)
+                           VALUES (?, ?, ?, ?, ?)""", (ven_id, event_id, None, 'new', serialized_event))
+
+async def _on_poll(ven_id, request_id=None):
+    cur = DB.cursor()
+    cur.execute("""SELECT event_id, event FROM events WHERE ven_id = ? AND status = 'new' LIMIT 1""", (ven_id,))
+    result = cur.fetchone()
+    if result:
+        event_id, event = result
+        event_request_id = generate_id()
+        with DB:
+            DB.execute("""UPDATE events SET request_id = ? WHERE event_id = ?""", (event_request_id, event_id))
+        response_type = 'oadrDistributeEvent'
+        response_payload = {'response': {'request_id': request_id,
+                                         'response_code': 200,
+                                         'response_description': 'OK'},
+                            'request_id': event_request_id,
+                            'vtn_id': VTN_ID,
+                            'events': [json.loads(event)]}
+    else:
+        response_type = 'oadrResponse'
+        response_payload = {'response': {'request_id': request_id,
+                                         'response_code': 200,
+                                         'response_description': 'OK'},
+                            'ven_id': ven_id}
+    return response_type, response_payload
 
 @pytest.mark.asyncio
-async def test_conformance_021(start_server):
+async def test_conformance_021():
     """
     If venID, vtnID, or eventID value is included in the payload, the receiving
     entity MUST validate that the ID value is as expected and generate an error
@@ -36,6 +116,10 @@ async def test_conformance_021(start_server):
     Exception: A VEN MUST NOT generate an error upon receipt of a canceled
     event whose eventID is not previously known.
     """
+    server = OpenADRServer(vtn_id='TestVTN', http_port=8001)
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+    server.add_handler('on_poll', _on_poll)
+    await server.run_async()
 
     client = OpenADRClient(ven_name="TestVEN",
                            vtn_url="http://localhost:8001/OpenADR2/Simple/2.0b")
@@ -72,3 +156,6 @@ async def test_conformance_021(start_server):
               event=event)
     message_type, message_payload = await client.poll()
     assert message_type == 'oadrDistributeEvent'
+    await client.stop()
+    await server.stop()
+

test/fixtures/simple_server.py

@@ -35,7 +35,7 @@ class EventFormatter(json.JSONEncoder):
             return timedeltaformat(obj)
         if isinstance(obj, datetime):
             return datetimeformat(obj)
-        if isinstance(obj, boolean):
+        if isinstance(obj, bool):
             return booleanformat(obj)
         return json.JSONEncoder.default(self, obj)
 
@@ -47,7 +47,7 @@ with DB:
 def lookup_ven(ven_name):
     with DB:
         DB.execute("SELECT * FROM vens WHERE ven_name = ?", (ven_name,))
-        ven = cur.fetchone()
+        ven = DB.fetchone()
     return ven
 
 def add_ven(ven_name, ven_id, registration_id):
@@ -100,17 +100,12 @@ async def _on_create_party_registration(payload):
     return 'oadrCreatedPartyRegistration', payload
 
 
-server = OpenADRServer(vtn_id=VTN_ID)
+server = OpenADRServer(vtn_id=VTN_ID, http_port=SERVER_PORT)
 server.add_handler('on_create_party_registration', _on_create_party_registration)
 server.add_handler('on_poll', _on_poll)
 
 @pytest.fixture
 async def start_server():
-    runner = web.AppRunner(server.app)
-    await runner.setup()
-    site = web.TCPSite(runner, 'localhost', SERVER_PORT)
-    await site.start()
-    print("SERVER IS NOW RUNNING")
+    await server.run_async()
     yield
-    print("SERVER IS NOW STOPPING")
-    await runner.cleanup()
+    await server.stop()

test/integration_tests/test_client_registration.py

@@ -31,45 +31,29 @@ VEN_NAME = 'myven'
 VEN_ID = '1234abcd'
 VTN_ID = "TestVTN"
 
-CERTFILE = os.path.join(os.path.dirname(os.path.dirname(__file__)), "cert.pem")
-KEYFILE =  os.path.join(os.path.dirname(os.path.dirname(__file__)), "key.pem")
+CERTFILE = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'certificates', 'dummy_ven.crt')
+KEYFILE =  os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'certificates', 'dummy_ven.key')
 
 
 async def _on_create_party_registration(payload):
     registration_id = generate_id()
-    payload = {'response': {'response_code': 200,
-                            'response_description': 'OK',
-                            'request_id': payload['request_id']},
-               'ven_id': VEN_ID,
-               'registration_id': registration_id,
-               'profiles': [{'profile_name': '2.0b',
-                             'transports': {'transport_name': 'simpleHttp'}}],
-               'requested_oadr_poll_freq': timedelta(seconds=10)}
-    return 'oadrCreatedPartyRegistration', payload
+    return VEN_ID, registration_id
 
 @pytest.fixture
 async def start_server():
-    server = OpenADRServer(vtn_id=VTN_ID)
+    server = OpenADRServer(vtn_id=VTN_ID, http_port=SERVER_PORT)
     server.add_handler('on_create_party_registration', _on_create_party_registration)
-
-    runner = web.AppRunner(server.app)
-    await runner.setup()
-    site = web.TCPSite(runner, 'localhost', SERVER_PORT)
-    await site.start()
+    await server.run_async()
     yield
-    await runner.cleanup()
+    await server.stop()
 
 @pytest.fixture
 async def start_server_with_signatures():
-    server = OpenADRServer(vtn_id=VTN_ID, cert=CERTFILE, key=KEYFILE, passphrase='openadr', fingerprint_lookup=fingerprint_lookup)
+    server = OpenADRServer(vtn_id=VTN_ID, cert=CERTFILE, key=KEYFILE, fingerprint_lookup=fingerprint_lookup, http_port=SERVER_PORT)
     server.add_handler('on_create_party_registration', _on_create_party_registration)
-
-    runner = web.AppRunner(server.app)
-    await runner.setup()
-    site = web.TCPSite(runner, 'localhost', SERVER_PORT)
-    await site.start()
+    await server.run_async()
     yield
-    await runner.cleanup()
+    await server.stop()
 
 
 @pytest.mark.asyncio
@@ -80,6 +64,7 @@ async def test_query_party_registration(start_server):
     response_type, response_payload = await client.query_registration()
     assert response_type == 'oadrCreatedPartyRegistration'
     assert response_payload['vtn_id'] == VTN_ID
+    await client.stop()
 
 @pytest.mark.asyncio
 async def test_create_party_registration(start_server):
@@ -89,7 +74,7 @@ async def test_create_party_registration(start_server):
     response_type, response_payload = await client.create_party_registration()
     assert response_type == 'oadrCreatedPartyRegistration'
     assert response_payload['ven_id'] == VEN_ID
-
+    await client.stop()
 
 def fingerprint_lookup(ven_id):
     with open(CERTFILE) as file:
@@ -102,10 +87,11 @@ async def test_create_party_registration_with_signatures(start_server_with_signa
         cert = file.read()
     client = OpenADRClient(ven_name=VEN_NAME,
                            vtn_url=f"http://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b",
-                           cert=CERTFILE, key=KEYFILE, passphrase='openadr', vtn_fingerprint=certificate_fingerprint(cert))
+                           cert=CERTFILE, key=KEYFILE, vtn_fingerprint=certificate_fingerprint(cert))
 
     response_type, response_payload = await client.create_party_registration()
     assert response_type == 'oadrCreatedPartyRegistration'
     assert response_payload['ven_id'] == VEN_ID
+    await client.stop()
 
 

test/integration_tests/test_event_warnings_errors.py

@@ -0,0 +1,399 @@
+from openleadr import OpenADRClient, OpenADRServer, enable_default_logging, utils, messaging
+import pytest
+from functools import partial
+import asyncio
+from datetime import datetime, timedelta, timezone
+import logging
+
+enable_default_logging()
+
+async def on_create_party_registration(ven_name):
+    return 'venid', 'regid'
+
+async def on_event_accepted(ven_id, event_id, opt_type, future=None):
+    if future and future.done() is False:
+        future.set_result(opt_type)
+
+async def good_on_event(event):
+    return 'optIn'
+
+async def faulty_on_event(event):
+    return None
+
+async def broken_on_event(event):
+    raise KeyError("BOOM")
+
+@pytest.mark.asyncio
+async def test_client_no_event_handler(caplog):
+    caplog.set_level(logging.WARNING)
+    logger = logging.getLogger('openleadr')
+    logger.setLevel(logging.DEBUG)
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')
+    server = OpenADRServer(vtn_id='myvtn', requested_poll_freq=timedelta(seconds=1))
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+    print("Running server")
+    await server.run_async()
+    # await asyncio.sleep(0.1)
+    print("Running client")
+    await client.run()
+
+    event_confirm_future = asyncio.get_event_loop().create_future()
+    print("Adding event")
+    server.add_event(ven_id='venid',
+                     event_id='test_client_no_event_handler',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[{'dtstart': datetime.now(timezone.utc),
+                                 'duration': timedelta(seconds=1),
+                                 'signal_payload': 1.1}],
+                     target={'ven_id': 'venid'},
+                     callback=partial(on_event_accepted, future=event_confirm_future))
+
+    print("Waiting for a response to the event")
+    result = await event_confirm_future
+    assert result == 'optOut'
+    assert ("You should implement your own on_event handler. This handler receives "
+            "an Event dict and should return either 'optIn' or 'optOut' based on your "
+            "choice. Will opt out of the event for now.") in [rec.message for rec in caplog.records]
+    await client.stop()
+    await server.stop()
+    await asyncio.gather(*[t for t in asyncio.all_tasks()][1:])
+
+@pytest.mark.asyncio
+async def test_client_faulty_event_handler(caplog):
+    caplog.set_level(logging.WARNING)
+    logger = logging.getLogger('openleadr')
+    logger.setLevel(logging.DEBUG)
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')
+    client.add_handler('on_event', faulty_on_event)
+    server = OpenADRServer(vtn_id='myvtn', requested_poll_freq=timedelta(seconds=1))
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+    print("Running server")
+    await server.run_async()
+    # await asyncio.sleep(0.1)
+    print("Running client")
+    await client.run()
+
+    event_confirm_future = asyncio.get_event_loop().create_future()
+    print("Adding event")
+    server.add_event(ven_id='venid',
+                     event_id='test_client_faulty_event_handler',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[{'dtstart': datetime.now(timezone.utc),
+                                 'duration': timedelta(seconds=1),
+                                 'signal_payload': 1.1}],
+                     target={'ven_id': 'venid'},
+                     callback=partial(on_event_accepted, future=event_confirm_future))
+
+    print("Waiting for a response to the event")
+    result = await event_confirm_future
+    assert result == 'optOut'
+    assert ("Your on_event or on_update_event handler must return 'optIn' or 'optOut'; "
+           f"you supplied {None}. Please fix your on_event handler.") in [rec.message for rec in caplog.records]
+    await client.stop()
+    await server.stop()
+    await asyncio.gather(*[t for t in asyncio.all_tasks()][1:])
+
+@pytest.mark.asyncio
+async def test_client_exception_event_handler(caplog):
+    caplog.set_level(logging.WARNING)
+    logger = logging.getLogger('openleadr')
+    logger.setLevel(logging.DEBUG)
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')
+    client.add_handler('on_event', broken_on_event)
+    server = OpenADRServer(vtn_id='myvtn', requested_poll_freq=timedelta(seconds=1))
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+    print("Running server")
+    await server.run_async()
+    # await asyncio.sleep(0.1)
+    print("Running client")
+    await client.run()
+
+    event_confirm_future = asyncio.get_event_loop().create_future()
+    print("Adding event")
+    server.add_event(ven_id='venid',
+                     event_id='test_client_exception_event_handler',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[{'dtstart': datetime.now(timezone.utc),
+                                 'duration': timedelta(seconds=1),
+                                 'signal_payload': 1.1}],
+                     target={'ven_id': 'venid'},
+                     callback=partial(on_event_accepted, future=event_confirm_future))
+
+    print("Waiting for a response to the event")
+    result = await event_confirm_future
+    assert result == 'optOut'
+
+    err = KeyError("BOOM")
+    assert ("Your on_event handler encountered an error. Will Opt Out of the event. "
+           f"The error was {err.__class__.__name__}: {str(err)}") in [rec.message for rec in caplog.records]
+    await client.stop()
+    await server.stop()
+    await asyncio.gather(*[t for t in asyncio.all_tasks()][1:])
+
+@pytest.mark.asyncio
+async def test_client_good_event_handler(caplog):
+    caplog.set_level(logging.WARNING)
+    logger = logging.getLogger('openleadr')
+    logger.setLevel(logging.DEBUG)
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')
+    client.add_handler('on_event', good_on_event)
+    server = OpenADRServer(vtn_id='myvtn', requested_poll_freq=timedelta(seconds=1))
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+    print("Running server")
+    await server.run_async()
+    # await asyncio.sleep(0.1)
+    print("Running client")
+    await client.run()
+
+    event_confirm_future = asyncio.get_event_loop().create_future()
+    print("Adding event")
+    server.add_event(ven_id='venid',
+                     event_id='test_client_good_event_handler',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[{'dtstart': datetime.now(timezone.utc),
+                                 'duration': timedelta(seconds=1),
+                                 'signal_payload': 1.1}],
+                     target={'ven_id': 'venid'},
+                     callback=partial(on_event_accepted, future=event_confirm_future))
+
+    print("Waiting for a response to the event")
+    result = await event_confirm_future
+    assert result == 'optIn'
+    assert len(caplog.records) == 0
+    await client.stop()
+    await server.stop()
+    # await asyncio.sleep(1)
+
+@pytest.mark.asyncio
+async def test_server_warning_conflicting_poll_methods(caplog):
+    caplog.set_level(logging.WARNING)
+    logger = logging.getLogger('openleadr')
+    logger.setLevel(logging.DEBUG)
+    server = OpenADRServer(vtn_id='myvtn', requested_poll_freq=timedelta(seconds=1))
+    server.add_handler('on_poll', print)
+    server.add_event(ven_id='venid',
+                     event_id='test_server_warning_conflicting_poll_methods',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[{'dtstart': datetime.now(timezone.utc),
+                                 'duration': timedelta(seconds=1),
+                                 'signal_payload': 1.1}],
+                     target={'ven_id': 'venid'},
+                     callback=on_event_accepted)
+    assert ("You cannot use the add_event method after you assign your own on_poll "
+            "handler. If you use your own on_poll handler, you are responsible for "
+            "delivering events from that handler. If you want to use OpenLEADRs "
+            "message queuing system, you should not assign an on_poll handler. "
+            "Your Event will NOT be added.") in [record.msg for record in caplog.records]
+
+
+@pytest.mark.asyncio
+async def test_server_warning_naive_datetimes_in_event(caplog):
+    caplog.set_level(logging.WARNING)
+    logger = logging.getLogger('openleadr')
+    logger.setLevel(logging.DEBUG)
+    server = OpenADRServer(vtn_id='myvtn', requested_poll_freq=timedelta(seconds=1))
+    server.add_event(ven_id='venid',
+                     event_id='test_server_warning_naive_datetimes_in_event',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[{'dtstart': datetime.now(),
+                                 'duration': timedelta(seconds=1),
+                                 'signal_payload': 1.1}],
+                     target={'ven_id': 'venid'},
+                     callback=on_event_accepted)
+    assert ("You supplied a naive datetime object to your interval's dtstart. "
+            "This will be interpreted as a timestamp in your local timezone "
+            "and then converted to UTC before sending. Please supply timezone-"
+            "aware timestamps like datetime.datetime.new(timezone.utc) or "
+            "datetime.datetime(..., tzinfo=datetime.timezone.utc)") in [record.msg for record in caplog.records]
+
+
+def test_event_with_wrong_response_required(caplog):
+    now = datetime.now(timezone.utc)
+    event = {'active_period': {'dtstart': now, 'duration': timedelta(seconds=10)},
+             'event_descriptor': {'event_id': 'event123',
+                                  'modification_number': 1,
+                                  'priority': 0,
+                                  'event_status': 'far',
+                                  'created_date_time': now},
+             'event_signals': [{'signal_name': 'simple',
+                                'signal_type': 'level',
+                                'intervals': [{'dtstart': now,
+                                               'duration': timedelta(seconds=10),
+                                               'signal_payload': 1}]}],
+             'targets': [{'ven_id': 'ven123'}],
+             'response_required': 'blabla'}
+    msg = messaging.create_message('oadrDistributeEvent', events=[event])
+    assert ("The response_required property in an Event should be "
+            "'never' or 'always', not blabla. Changing to 'always'.") in caplog.messages
+    message_type, message_payload= messaging.parse_message(msg)
+    assert message_payload['events'][0]['response_required'] == 'always'
+
+
+def test_event_missing_created_date_time(caplog):
+    now = datetime.now(timezone.utc)
+    event = {'active_period': {'dtstart': now, 'duration': timedelta(seconds=10)},
+             'event_descriptor': {'event_id': 'event123',
+                                  'modification_number': 1,
+                                  'priority': 0,
+                                  'event_status': 'far'},
+             'event_signals': [{'signal_name': 'simple',
+                                'signal_type': 'level',
+                                'intervals': [{'dtstart': now,
+                                               'duration': timedelta(seconds=10),
+                                               'signal_payload': 1}]}],
+             'targets': [{'ven_id': 'ven123'}],
+             'response_required': 'always'}
+    msg = messaging.create_message('oadrDistributeEvent', events=[event])
+    assert ("Your event descriptor did not contain a created_date_time. "
+            "This will be automatically added.") in caplog.messages
+
+
+def test_event_incongruent_targets(caplog):
+    now = datetime.now(timezone.utc)
+    event = {'active_period': {'dtstart': now, 'duration': timedelta(seconds=10)},
+             'event_descriptor': {'event_id': 'event123',
+                                  'modification_number': 1,
+                                  'priority': 0,
+                                  'event_status': 'far',
+                                  'created_date_time': now},
+             'event_signals': [{'signal_name': 'simple',
+                                'signal_type': 'level',
+                                'intervals': [{'dtstart': now,
+                                               'duration': timedelta(seconds=10),
+                                               'signal_payload': 1}]}],
+             'targets': [{'ven_id': 'ven123'}],
+             'targets_by_type': {'ven_id': ['ven456']},
+             'response_required': 'always'}
+    with pytest.raises(ValueError) as err:
+        msg = messaging.create_message('oadrDistributeEvent', events=[event])
+    assert str(err.value) == ("You assigned both 'targets' and 'targets_by_type' in your event, "
+                "but the two were not consistent with each other. "
+                f"You supplied 'targets' = {event['targets']} and "
+                f"'targets_by_type' = {event['targets_by_type']}")
+
+
+def test_event_only_targets_by_type(caplog):
+    now = datetime.now(timezone.utc)
+    event = {'active_period': {'dtstart': now, 'duration': timedelta(seconds=10)},
+             'event_descriptor': {'event_id': 'event123',
+                                  'modification_number': 1,
+                                  'priority': 0,
+                                  'event_status': 'far',
+                                  'created_date_time': now},
+             'event_signals': [{'signal_name': 'simple',
+                                'signal_type': 'level',
+                                'intervals': [{'dtstart': now,
+                                               'duration': timedelta(seconds=10),
+                                               'signal_payload': 1}]}],
+             'targets_by_type': {'ven_id': ['ven456']},
+             'response_required': 'always'}
+    msg = messaging.create_message('oadrDistributeEvent', events=[event])
+    message_type, message_payload = messaging.parse_message(msg)
+    assert message_payload['events'][0]['targets'] == [{'ven_id': 'ven456'}]
+
+@pytest.mark.asyncio
+async def test_client_warning_no_update_event_handler(caplog):
+    caplog.set_level(logging.WARNING)
+    logger = logging.getLogger('openleadr')
+    logger.setLevel(logging.DEBUG)
+    server = OpenADRServer(vtn_id='myvtn', requested_poll_freq=timedelta(seconds=1))
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+    event_accepted_future = asyncio.get_event_loop().create_future()
+    server.add_event(ven_id='venid',
+                     event_id='test_client_warning_no_update_event_handler',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[{'dtstart': datetime.now(timezone.utc),
+                                 'duration': timedelta(seconds=1),
+                                 'signal_payload': 1.1}],
+                     target={'ven_id': 'venid'},
+                     callback=event_accepted_future)
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')
+    client.add_handler('on_event', good_on_event)
+    print("Starting server")
+    await server.run()
+    await client.run()
+    print("Waiting for first event to be accepted...")
+    await event_accepted_future
+
+    # Manually update the event
+    server.events['venid'][0].event_descriptor.modification_number = 1
+    server.events_updated['venid'] = True
+
+    await asyncio.sleep(1)
+    assert ("You should implement your own on_update_event handler. This handler receives "
+            "an Event dict and should return either 'optIn' or 'optOut' based on your "
+            "choice. Will re-use the previous opt status for this event_id for now") in [record.msg for record in caplog.records]
+    await client.stop()
+    await server.stop()
+
+@pytest.mark.asyncio
+async def test_server_add_event_with_wrong_callback_signature(caplog):
+    def dummy_callback(some_param):
+        pass
+    caplog.set_level(logging.WARNING)
+    logger = logging.getLogger('openleadr')
+    logger.setLevel(logging.DEBUG)
+    server = OpenADRServer(vtn_id='myvtn', requested_poll_freq=timedelta(seconds=1))
+    with pytest.raises(ValueError) as err:
+        server.add_event(ven_id='venid',
+                         event_id='test_server_add_event_with_wrong_callback_signature',
+                         signal_name='simple',
+                         signal_type='level',
+                         intervals=[{'dtstart': datetime.now(timezone.utc),
+                                     'duration': timedelta(seconds=1),
+                                     'signal_payload': 1.1}],
+                         target={'ven_id': 'venid'},
+                         callback=dummy_callback)
+
+@pytest.mark.asyncio
+async def test_server_add_event_with_no_callback(caplog):
+    def dummy_callback(some_param):
+        pass
+    caplog.set_level(logging.WARNING)
+    logger = logging.getLogger('openleadr')
+    logger.setLevel(logging.DEBUG)
+    server = OpenADRServer(vtn_id='myvtn')
+    server.add_event(ven_id='venid',
+                     event_id='test_server_add_event_with_no_callback',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[{'dtstart': datetime.now(timezone.utc),
+                                 'duration': timedelta(seconds=1),
+                                 'signal_payload': 1.1}],
+                     target={'ven_id': 'venid'})
+    assert ("You did not provide a 'callback', which means you won't know if the "
+            "VEN will opt in or opt out of your event. You should consider adding "
+            "a callback for this.") in caplog.messages
+
+@pytest.mark.asyncio
+async def test_server_add_event_with_no_callback_response_never_required(caplog):
+    caplog.set_level(logging.WARNING)
+    logger = logging.getLogger('openleadr')
+    logger.setLevel(logging.DEBUG)
+    server = OpenADRServer(vtn_id='myvtn')
+    server.add_event(ven_id='venid',
+                     event_id='test_server_add_event_with_no_callback_response_never_required',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[{'dtstart': datetime.now(timezone.utc),
+                                 'duration': timedelta(seconds=1),
+                                 'signal_payload': 1.1}],
+                     target={'ven_id': 'venid'},
+                     response_required='never')
+    await server.run()
+    await server.stop()
+    assert ("You did not provide a 'callback', which means you won't know if the "
+            "VEN will opt in or opt out of your event. You should consider adding "
+            "a callback for this.") not in caplog.messages

test/key.pem

@@ -1,54 +0,0 @@
------BEGIN ENCRYPTED PRIVATE KEY-----
-MIIJnDBOBgkqhkiG9w0BBQ0wQTApBgkqhkiG9w0BBQwwHAQIx0KwpLX3E6ECAggA
-MAwGCCqGSIb3DQIJBQAwFAYIKoZIhvcNAwcECJ//ErIFZQJJBIIJSKJgQttwaFke
-9rBpQLmpQgWVYt9XTcOmJNXnOAZdrRv0JzOoo2LYR+ejDSWFJyXznL3TLr4vywkn
-YvPM1QFIlt7EKWb5IeEe7zZa9aWAHa2rFq74a0mltzkggn99WWodB/IBDKlBOnHp
-dyw6qGDzwrTREvzdVwDF8hqRUNp6AccO9BGWgxr6ipdbvDUdHqUSb6HMle0hrQ7T
-P6wtaRr2OaRVJEwVGwFCW/lymMx+kXf7u5GnZyVfB/eMfe/iDu3hfj55m8RVWHZR
-LGSaTT7kkpsCjLPGzQwbz7qwxdGJ27K6Cil0J2eqpihcolTef+qNqUOWvmtkNS9r
-VxtFp4CtFG28Vz3OtqlP4QeZ2JDFUeHbvtCSn/Q8shlAo1PfxgDIf4vMXI1LVHok
-D8Cd//CczX3gIt/6V9eLDgb6nX6ozPnCond4o5aMp5VIt1/YQvYBROvhvCeCdAUh
-imfWmAwb7zJltFizuMek9kwpTnYt+3ZsA6gSTHTsS301U4ne0h1YEyL3SO9/gWeu
-m2vTEmefDY1vHp2EaHxFcWA3q3yyCoLQ0BvOBcgStnXzoM3Qo9d73obTizt950Vx
-jhLfBYRxoXTyJbIKajDU/apkSyT8BeiWuahkv+zvHt7fnTLwaJ42L+4WpCFRpQv+
-pjUiw8gX7ZCqtq3FC9xYQyHKV/6NDIc1evmtIXHBtS3uiMCm2Hxy6YOlQeOtGv4g
-QiloWHPi4fWz/nQ6RxUNZh3F1eKtTxPxInqKjRYO8T553r7mr45+xd5ldI3GZkND
-3eDIDeRdSqotLrbQrNZ9690J7VCpXnlwjvg4rsa4jQaCJG4Kyj7AdIUKlVJ+m2Pg
-f6UkYjJDpnl0eJ94HOa9fIHYnvXvzYc8YhEbfAGGvHMLV+L8fgE/t2rcu0WBZlUt
-QprcN8KNuaVrYJHxF4UYE6tcdu+4LzqJpZRY0JKmsryBi/URO9R5fpzOqMm/Hnk+
-OWo0mUqfJvIIKPpa7JNhk8P7vwUS5SLB+9kHJkYBRPQcnzUh3/ii6hR//8ogkW9u
-KaRd9QMANzO6f0MsgvxjqbqwNFVFCNDA6ZMF8wCeZDjl5ompxvD2ED50SUgwjKLf
-iInnHbe7TD02oBD0EzfR22d0+36Q+gsKV5xW1O+qhx159uvkgAVsxzqD1fa6ylBV
-ZNmmxfH8RwRiFimpvTluknwVQxUIGEsnXy3ubTS4W4J1alj5GQTvM2lGRha4HvHZ
-pT3y+1rSxcBHEA4iiKoAlRu+dUYjDaD5+39rvDLXev234w2dzise+H2ppaNIewLE
-UlOAs80zdLioiM6IVLnw2m1mJGnYbkkP+o/PKEjumnTvnMMbnl6CEXLWIucT8Okf
-/Jbe8NAS9lu4+1jnpImYewjNRuJbQsMHariIcjUNHW/8wUBA/Y7wOHExTw/vvY82
-xm6ZvfsN59U4UZPkMZpCrCqNh3USwhT4671+LlBNHiRFeddbm/Ko+6EgRiK1JCUB
-SlneKQybezlsWn+/hu3Ub+j+2qzBjcUnRSjLsv9KZem7Yz4FLAISwXK15iq2Z+7M
-FwKSZ12QjXjTPzmBXGRQaIQrccIWmFmgJ5mQ2T+oH0ucWFWO49vLvuh4oJ0H4kop
-xGNgqC/8HD61REtbADdQNwGwzdn5brVzohQ+8tTPWjsqGEqwyofIp/L7pavENJKg
-ho6RqWqa2CNA/dppwKQH6bO+X8CUQissOyCBovYqvKMwuRAsYwD2J5T64Zso+WTm
-3njYGfOS8hoyfjrW7qX3Pcxm5fBntxbeKbdgKzeCxNgVxdkiYpQ+h1CxFWArqvRo
-myAstRzSFNGGuC81q0Gm9F71mnD93Zmc1F/3np9A/WdzJ6YL0iti2lVwseCFMQjU
-8bQ/KxoBw4xWp1isy5xFilca7psVSwV+hnFaOxKf1CwL7ZEQpJaGcUtl5p16AGwW
-kezn+pDfK3SIvsZd3OS7mfrgbkeXcxQi6Dcfvdtvg+OvFGOdRwurKkPWIvegVWeb
-nkssQNTiQsEmxjnFdyPqNDB0ilL9viEwaPEchYZmKWVhnn6OwjvW0hwgx4YIb23k
-5P7jwlcyZLFwuhm4wahd7Veh9fEYQlaKpufXaMtUNkX+x4ts6Cy9Ap3qhjVp1JxK
-Vj5yjCCQBLsGUZ0CDjI0Oph3z54BrFAbbqVdedvypmEY1SXDYZ0d4RdcuUohkPDZ
-zJ74gZslR5qV6JkXGjZZUCvw4Ea49ZxCR5XhBrIqfuhfRrdPd4Quoh8LCUPlRa+q
-kAhjKVXliVOLL/yCM5jwqrnHhxGp5jBa32lQAfZHdkVTlG7xAjlCyTJg+n90gejN
-4m2hanUPC3C4qv4Fj0sdWrty9mPL5sKUmFT/RMlfEG8Y9DZXwx2nXWPMYsScO3qP
-Q/+u2G2Spj4ijuWZpfPmikqXIWQYHUEpeLiX38eEDnn4kGkT13aT4KzwUnISVQPi
-f+yIlqGFWPlS7ZwUh0jsq6UEzJ6CyjHwkxMRlLH8VseJOZlmlJqIq9K+59gZWaO2
-fFhqQHgrJIJEMIf+5e37n1SWUsfWkTDkyMtPa43QYV6d8Vh/c+mjRiApxDu3EZpg
-kgGc7uLESaht0FlUWs1p8Dn07WqSxc3qwzK3nv+XgLSR0XFbU+lar5SUp4PdjOKW
-QFmoKo77hjRFgVFh5FW+ZY1T4827yz2LgplQ1IMkx+fWT+rPEQOZfMOWzKkdc3DF
-55iTW0UglsKbS9j79JaXcIGOo7Wx1AvMFUijbbYei4PV2+GjWahKFXE/cClrlG8S
-+oM+V1Y0SCX39CT6wRKMgeOCmtw/9wPFtoxPSAcet2V6ZbzkwEskAl7/xp+WpX6e
-TUpspg7bqhnUHsIBh5NI65INdYZbAA55ykuqMT01ZgxkFRyhymFBf32nE2dQ1UxX
-n95musHsifY8URg+0o9kGvlGzV7Yx98F9rojvd9oLnrmMjRpYEcrc42XwbNlVwx6
-Z7N4qIz8rKxYWWrAhg9BY4NLKQzmsMeVtKXgUVni2Vjv7cKqiAmAw1gBVhIvJLYe
-GVJwuZCOosmF31DNVQOU4XOTWNhjTpg0/jCbLtnrmiVcb70pGrKv4a3v3LyS59i6
-ohm0oZui3W+utcO4fPOlWhctui8OquUxrm2dWbfOPIa8yndNSvLWRKs6GAukaK+9
-J/RQZbipAoMuY1b7r9U/hQ==
------END ENCRYPTED PRIVATE KEY-----

test/test_certificates.py

@@ -0,0 +1,99 @@
+import asyncio
+import pytest
+import os
+from functools import partial
+from openleadr import OpenADRServer, OpenADRClient, enable_default_logging
+from openleadr.utils import certificate_fingerprint
+from openleadr import errors
+from async_timeout import timeout
+
+enable_default_logging()
+
+CA_CERT = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'certificates', 'dummy_ca.crt')
+VTN_CERT = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'certificates', 'dummy_vtn.crt')
+VTN_KEY = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'certificates', 'dummy_vtn.key')
+VEN_CERT = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'certificates', 'dummy_ven.crt')
+VEN_KEY = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'certificates', 'dummy_ven.key')
+
+
+with open(VEN_CERT) as file:
+    ven_fingerprint = certificate_fingerprint(file.read())
+
+with open(VTN_CERT) as file:
+    vtn_fingerprint = certificate_fingerprint(file.read())
+
+async def lookup_fingerprint(ven_id):
+    return ven_fingerprint
+
+async def on_create_party_registration(payload, future):
+    if payload['fingerprint'] != ven_fingerprint:
+        raise errors.FingerprintMismatch("The fingerprint of your TLS connection does not match the expected fingerprint. Your VEN is not allowed to register.")
+    else:
+        future.set_result(True)
+        return 'ven1234', 'reg5678'
+
+@pytest.mark.asyncio
+async def test_ssl_certificates():
+    loop = asyncio.get_event_loop()
+    registration_future = loop.create_future()
+    server = OpenADRServer(vtn_id='myvtn',
+                           http_cert=VTN_CERT,
+                           http_key=VTN_KEY,
+                           http_ca_file=CA_CERT,
+                           cert=VTN_CERT,
+                           key=VTN_KEY,
+                           fingerprint_lookup=lookup_fingerprint)
+    server.add_handler('on_create_party_registration', partial(on_create_party_registration,
+                                                               future=registration_future))
+    await server.run_async()
+    #await asyncio.sleep(1)
+    # Run the client
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url='https://localhost:8080/OpenADR2/Simple/2.0b',
+                           cert=VEN_CERT,
+                           key=VEN_KEY,
+                           ca_file=CA_CERT,
+                           vtn_fingerprint=vtn_fingerprint)
+    await client.run()
+
+    # Wait for the registration to be triggered
+    result = await asyncio.wait_for(registration_future, 1.0)
+    assert client.registration_id == 'reg5678'
+
+    await client.stop()
+    await server.stop()
+    #await asyncio.sleep(0)
+
+@pytest.mark.asyncio
+async def test_ssl_certificates_wrong_cert():
+    loop = asyncio.get_event_loop()
+    registration_future = loop.create_future()
+    server = OpenADRServer(vtn_id='myvtn',
+                           http_cert=VTN_CERT,
+                           http_key=VTN_KEY,
+                           http_ca_file=CA_CERT,
+                           cert=VTN_CERT,
+                           key=VTN_KEY,
+                           fingerprint_lookup=lookup_fingerprint)
+    server.add_handler('on_create_party_registration', partial(on_create_party_registration,
+                                                               future=registration_future))
+    await server.run_async()
+    #await asyncio.sleep(1)
+
+    # Run the client
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url='https://localhost:8080/OpenADR2/Simple/2.0b',
+                           cert=VTN_CERT,
+                           key=VTN_KEY,
+                           ca_file=CA_CERT,
+                           vtn_fingerprint=vtn_fingerprint)
+    await client.run()
+
+    # Wait for the registration to be triggered
+    with pytest.raises(asyncio.TimeoutError):
+        await asyncio.wait_for(registration_future, timeout=0.5)
+    assert client.registration_id is None
+
+    await client.stop()
+    await server.stop()
+    await asyncio.sleep(0)

test/test_client_misc.py

@@ -0,0 +1,83 @@
+import pytest
+from openleadr import OpenADRClient
+from openleadr import enums
+
+def test_trailing_slash_on_vtn_url():
+    client = OpenADRClient(ven_name='myven', vtn_url='http://localhost/')
+    assert client.vtn_url == 'http://localhost'
+
+def test_wrong_handler_supplied(caplog):
+    client = OpenADRClient(ven_name='myven', vtn_url='http://localhost')
+    client.add_handler('non_existant', print)
+    assert ("'handler' must be either on_event or on_update_event") in [rec.message for rec in caplog.records]
+
+def test_invalid_report_name(caplog):
+    client = OpenADRClient(ven_name='myven', vtn_url='http://localhost')
+    with pytest.raises(ValueError):
+        client.add_report(callback=print,
+                          resource_id='myresource',
+                          measurement='voltage',
+                          report_name='non_existant')
+        # assert (f"non_existant is not a valid report_name. Valid options are "
+        #         f"{', '.join(enums.REPORT_NAME.values)}",
+        #         " or any name starting with 'x-'.") in [rec.message for rec in caplog.records]
+
+def test_invalid_reading_type():
+    client = OpenADRClient(ven_name='myven', vtn_url='http://localhost')
+    with pytest.raises(ValueError):
+        client.add_report(callback=print,
+                          resource_id='myresource',
+                          measurement='voltage',
+                          reading_type='non_existant')
+            # assert (f"non_existant is not a valid reading_type. Valid options are "
+            # f"{', '.join(enums.READING_TYPE.values)}",
+            # " or any name starting with 'x-'.") in [rec.message for rec in caplog.records]
+
+def test_invalid_report_type():
+    client = OpenADRClient(ven_name='myven', vtn_url='http://localhost')
+    with pytest.raises(ValueError):
+        client.add_report(callback=print,
+                          resource_id='myresource',
+                          measurement='voltage',
+                          report_type='non_existant')
+        # assert (f"non_existant is not a valid report_type. Valid options are "
+        #         f"{', '.join(enums.REPORT_TYPE.values)}",
+        #         " or any name starting with 'x-'.") in [rec.message for rec in caplog.records]
+
+def test_invalid_data_collection_mode():
+    client = OpenADRClient(ven_name='myven', vtn_url='http://localhost')
+    with pytest.raises(ValueError):
+        client.add_report(callback=print,
+                          resource_id='myresource',
+                          measurement='voltage',
+                          data_collection_mode='non_existant')
+        # assert ("The data_collection_mode should be 'incremental' or 'full'.") in [rec.message for rec in caplog.records]
+
+def test_invalid_scale():
+    client = OpenADRClient(ven_name='myven', vtn_url='http://localhost')
+    with pytest.raises(ValueError):
+        client.add_report(callback=print,
+                          resource_id='myresource',
+                          measurement='voltage',
+                          scale='non_existant')
+
+def test_add_report_without_specifier_id():
+    client = OpenADRClient(ven_name='myven', vtn_url='http://localhost')
+    client.add_report(callback=print,
+                      resource_id='myresource1',
+                      measurement='voltage')
+    client.add_report(callback=print,
+                      resource_id='myresource2',
+                      measurement='voltage')
+    assert len(client.reports) == 1
+
+async def wrong_sig(param1):
+    pass
+
+def test_add_report_with_invalid_callback_signature():
+    client = OpenADRClient(ven_name='myven', vtn_url='http://localhost')
+    with pytest.raises(TypeError):
+        client.add_report(callback=wrong_sig,
+                          data_collection_mode='full',
+                          resource_id='myresource1',
+                          measurement='voltage')

test/test_errors.py

@@ -0,0 +1,11 @@
+from openleadr import errors, enums
+
+def test_protocol_errors():
+    for error in dir(errors):
+        if isinstance(getattr(errors, error), type):
+            err = getattr(errors, error)()
+            if isinstance(err, errors.ProtocolError) and not type(err) == errors.ProtocolError:
+                err_description = err.response_description
+                err_code = err.response_code
+                err_enum = err_description.replace(" ", "_")
+                assert enums.STATUS_CODES[err_enum] == err_code

test/test_event_distribution.py

@@ -0,0 +1,286 @@
+from openleadr import OpenADRClient, OpenADRServer, enable_default_logging, objects, utils
+import pytest
+import asyncio
+import datetime
+from functools import partial
+
+enable_default_logging()
+
+def on_create_party_registration(registration_info):
+    print("Registered party")
+    return 'ven123', 'reg123'
+
+async def on_event(event):
+    return 'optIn'
+
+async def on_event_opt_in(event, future=None):
+    if future and future.done() is False:
+        future.set_result(event)
+    return 'optIn'
+
+async def on_update_event(event, futures):
+    for future in futures:
+        if future.done() is False:
+            future.set_result(event)
+            break
+    return 'optIn'
+
+async def on_event_opt_out(event, futures):
+    for future in futures:
+        if future.done() is False:
+            future.set_result(event)
+            break
+    return 'optOut'
+
+async def event_callback(ven_id, event_id, opt_type, future):
+    if future.done() is False:
+        future.set_result(opt_type)
+
+@pytest.mark.asyncio
+async def test_internal_message_queue():
+    loop = asyncio.get_event_loop()
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')
+    client.add_handler('on_event', on_event)
+    server = OpenADRServer(vtn_id='myvtn', requested_poll_freq=datetime.timedelta(seconds=1))
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+    event_callback_future = loop.create_future()
+    server.add_event(ven_id='ven123',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[{'dtstart': datetime.datetime.now(datetime.timezone.utc),
+                                 'duration': datetime.timedelta(seconds=3),
+                                 'signal_payload': 1}],
+                     callback=partial(event_callback, future=event_callback_future))
+
+    await server.run_async()
+    #await asyncio.sleep(1)
+    await client.run()
+    #await asyncio.sleep(1)
+    status = await event_callback_future
+    assert status == 'optIn'
+
+    message_type, message_payload = await asyncio.wait_for(client.poll(), 0.5)
+    assert message_type == 'oadrResponse'
+
+    message_type, message_payload = await asyncio.wait_for(client.poll(), 0.5)
+    assert message_type == 'oadrResponse'
+
+    #await asyncio.sleep(1)  # Wait for the event to be completed
+    await client.stop()
+    await server.stop()
+
+@pytest.mark.asyncio
+async def test_request_event():
+    loop = asyncio.get_event_loop()
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')
+
+    server = OpenADRServer(vtn_id='myvtn', requested_poll_freq=datetime.timedelta(seconds=1))
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+
+    event_id = server.add_event(ven_id='ven123',
+                                signal_name='simple',
+                                signal_type='level',
+                                intervals=[{'dtstart': datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(seconds=4),
+                                            'duration': datetime.timedelta(seconds=2),
+                                            'signal_payload': 1}],
+                                ramp_up_period=datetime.timedelta(seconds=2),
+                                callback=partial(event_callback))
+
+    assert server.events['ven123'][0].event_descriptor.event_status == 'far'
+    await server.run_async()
+    await client.create_party_registration()
+    message_type, message_payload = await client.request_event()
+    assert message_type == 'oadrDistributeEvent'
+    message_type, message_payload = await client.request_event()
+    assert message_type == 'oadrDistributeEvent'
+    await client.stop()
+    await server.stop()
+
+
+@pytest.mark.asyncio
+async def test_raw_event():
+    now = datetime.datetime.now(datetime.timezone.utc)
+    server = OpenADRServer(vtn_id='myvtn')
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+    event = objects.Event(event_descriptor=objects.EventDescriptor(event_id='event001',
+                                                                   modification_number=0,
+                                                                   event_status='far',
+                                                                   market_context='http://marketcontext01'),
+                          event_signals=[objects.EventSignal(signal_id='signal001',
+                                                             signal_type='level',
+                                                             signal_name='simple',
+                                                             intervals=[objects.Interval(dtstart=now,
+                                                                                         duration=datetime.timedelta(minutes=10),
+                                                                                         signal_payload=1)]),
+                                        objects.EventSignal(signal_id='signal002',
+                                                            signal_type='price',
+                                                            signal_name='ELECTRICITY_PRICE',
+                                                            intervals=[objects.Interval(dtstart=now,
+                                                                                        duration=datetime.timedelta(minutes=10),
+                                                                                        signal_payload=1)])],
+                          targets=[objects.Target(ven_id='ven123')])
+    loop = asyncio.get_event_loop()
+    event_callback_future = loop.create_future()
+    server.add_raw_event(ven_id='ven123', event=event, callback=partial(event_callback, future=event_callback_future))
+
+    on_event_future = loop.create_future()
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')
+    client.add_handler('on_event', partial(on_event_opt_in, future=on_event_future))
+
+    await server.run_async()
+    await client.run()
+    event = await on_event_future
+    assert len(event['event_signals']) == 2
+
+    result = await event_callback_future
+    assert result == 'optIn'
+
+    await client.stop()
+    await server.stop()
+
+
+@pytest.mark.asyncio
+async def test_create_event_with_future_as_callback():
+    now = datetime.datetime.now(datetime.timezone.utc)
+    server = OpenADRServer(vtn_id='myvtn')
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+    event = objects.Event(event_descriptor=objects.EventDescriptor(event_id='event001',
+                                                                   modification_number=0,
+                                                                   event_status='far',
+                                                                   market_context='http://marketcontext01'),
+                          event_signals=[objects.EventSignal(signal_id='signal001',
+                                                             signal_type='level',
+                                                             signal_name='simple',
+                                                             intervals=[objects.Interval(dtstart=now,
+                                                                                         duration=datetime.timedelta(minutes=10),
+                                                                                         signal_payload=1)]),
+                                        objects.EventSignal(signal_id='signal002',
+                                                            signal_type='price',
+                                                            signal_name='ELECTRICITY_PRICE',
+                                                            intervals=[objects.Interval(dtstart=now,
+                                                                                        duration=datetime.timedelta(minutes=10),
+                                                                                        signal_payload=1)])],
+                          targets=[objects.Target(ven_id='ven123')])
+    loop = asyncio.get_event_loop()
+    event_callback_future = loop.create_future()
+    server.add_raw_event(ven_id='ven123', event=event, callback=event_callback_future)
+
+    on_event_future = loop.create_future()
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')
+    client.add_handler('on_event', partial(on_event_opt_in, future=on_event_future))
+
+    await server.run_async()
+    await client.run()
+    event = await on_event_future
+    assert len(event['event_signals']) == 2
+
+    result = await event_callback_future
+    assert result == 'optIn'
+    await client.stop()
+    await server.stop()
+
+@pytest.mark.asyncio
+async def test_multiple_events_in_queue():
+    now = datetime.datetime.now(datetime.timezone.utc)
+    server = OpenADRServer(vtn_id='myvtn')
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+
+    loop = asyncio.get_event_loop()
+    event_1_callback_future = loop.create_future()
+    event_2_callback_future = loop.create_future()
+    server.add_event(ven_id='ven123',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[objects.Interval(dtstart=now,
+                                                 duration=datetime.timedelta(seconds=1),
+                                                 signal_payload=1)],
+                     callback=event_1_callback_future)
+
+    await server.run()
+
+    on_event_future = loop.create_future()
+    client = OpenADRClient(ven_name='ven123',
+                           vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')
+    await client.create_party_registration()
+    response_type, response_payload = await client.request_event()
+    assert response_type == 'oadrDistributeEvent'
+    events = response_payload['events']
+    assert len(events) == 1
+    event_id = events[0]['event_descriptor']['event_id']
+    request_id = response_payload['request_id']
+    await client.created_event(request_id=request_id,
+                               event_id=event_id,
+                               opt_type='optIn',
+                               modification_number=0)
+
+    server.add_event(ven_id='ven123',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[objects.Interval(dtstart=now + datetime.timedelta(seconds=1),
+                                                 duration=datetime.timedelta(seconds=1),
+                                                 signal_payload=1)],
+                     callback=event_2_callback_future)
+    response_type, response_payload = await client.request_event()
+    assert response_type == 'oadrDistributeEvent'
+    events = response_payload['events']
+
+    # Assert that we still have two events in the response
+    assert len(events) == 2
+
+    # Wait one second and retrieve the events again
+    await asyncio.sleep(1)
+    response_type, response_payload = await client.request_event()
+    assert response_type == 'oadrDistributeEvent'
+    events = response_payload['events']
+    assert len(events) == 2
+    assert events[1]['event_descriptor']['event_status'] == 'completed'
+
+    response_type, response_payload = await client.request_event()
+    assert response_type == 'oadrDistributeEvent'
+    events = response_payload['events']
+    assert len(events) == 1
+    await asyncio.sleep(1)
+
+    response_type, response_payload = await client.request_event()
+    assert response_type == 'oadrDistributeEvent'
+
+    response_type, response_payload = await client.request_event()
+    assert response_type == 'oadrResponse'
+
+    await server.stop()
+
+@pytest.mark.asyncio
+async def test_client_event_cleanup():
+    now = datetime.datetime.now(datetime.timezone.utc)
+    server = OpenADRServer(vtn_id='myvtn')
+    server.add_handler('on_create_party_registration', on_create_party_registration)
+
+    loop = asyncio.get_event_loop()
+    event_1_callback_future = loop.create_future()
+    event_2_callback_future = loop.create_future()
+    server.add_event(ven_id='ven123',
+                     signal_name='simple',
+                     signal_type='level',
+                     intervals=[objects.Interval(dtstart=now,
+                                                 duration=datetime.timedelta(seconds=1),
+                                                 signal_payload=1)],
+                     callback=event_1_callback_future)
+    await server.run()
+
+    client = OpenADRClient(ven_name='ven123',
+                           vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')
+    client.add_handler('on_event', on_event_opt_in)
+    await client.run()
+    await asyncio.sleep(0.5)
+    assert len(client.received_events) == 1
+
+    await asyncio.sleep(0.5)
+    await client._event_cleanup()
+    assert len(client.received_events) == 0
+
+    await server.stop()
+    await client.stop()

test/test_failures.py

@@ -1,31 +1,220 @@
 from openleadr import OpenADRClient, OpenADRServer
-from openleadr.utils import generate_id
+from openleadr.utils import generate_id, certificate_fingerprint
+from openleadr import messaging, errors
 import pytest
 from aiohttp import web
 import os
+import logging
 import asyncio
 from datetime import timedelta
+from base64 import b64encode
+import re
+from lxml import etree
 
 @pytest.mark.asyncio
 async def test_http_level_error(start_server):
     client = OpenADRClient(vtn_url="http://this.is.an.error", ven_name=VEN_NAME)
     client.on_event = _client_on_event
     await client.run()
+    await client.client_session.close()
+
 
 @pytest.mark.asyncio
-async def test_openadr_error(start_server):
-    client = OpenADRClient(vtn_url=f"http://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b", ven_name=VEN_NAME)
-    client.on_event = _client_on_event
+async def test_xml_schema_error(start_server, caplog):
+    message = messaging.create_message("oadrQueryRegistration", request_id='req1234')
+    message = message.replace('<requestID xmlns="http://docs.oasis-open.org/ns/energyinterop/201110/payloads">req1234</requestID>', '')
+    client = OpenADRClient(ven_name='myven', vtn_url=f'http://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b')
+    result = await client._perform_request('EiRegisterParty', message)
+    assert result == (None, {})
+
+    logs = [rec.message for rec in caplog.records]
+    for log in logs:
+        if log.startswith("Non-OK status 400"):
+          assert "XML failed validation" in log
+          break
+    else:
+        assert False
+
+@pytest.mark.asyncio
+async def test_wrong_endpoint(start_server, caplog):
+    message = messaging.create_message("oadrQueryRegistration", request_id='req1234')
+    client = OpenADRClient(ven_name='myven', vtn_url=f'http://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b')
+    response_type, response_payload = await client._perform_request('OadrPoll', message)
+    assert response_type == 'oadrResponse'
+    assert response_payload['response']['response_code'] == 459
+
+@pytest.mark.asyncio
+async def test_vtn_no_create_party_registration_handler(caplog):
+    caplog.set_level(logging.WARNING)
+    server = OpenADRServer(vtn_id='myvtn')
+    client = OpenADRClient(ven_name='myven', vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')
+    await server.run_async()
     await client.run()
+    #await asyncio.sleep(0.5)
+    await server.stop()
+    await client.stop()
+    #await asyncio.sleep(0)
+    assert 'No VEN ID received from the VTN, aborting.' in caplog.messages
+    assert ("You should implement and register your own on_create_party_registration "
+            "handler if you want VENs to be able to connect to you. This handler will "
+            "receive a registration request and should return either 'False' (if the "
+            "registration is denied) or a (ven_id, registration_id) tuple if the "
+            "registration is accepted.") in caplog.messages
+
+@pytest.mark.asyncio
+async def test_invalid_signature_error(start_server_with_signatures, caplog):
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url=f'https://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b',
+                           cert=VEN_CERT,
+                           key=VEN_KEY,
+                           vtn_fingerprint=VTN_FINGERPRINT)
+    message = client._create_message('oadrPoll', ven_id='ven123')
+    fake_sig = b64encode("HelloThere".encode('utf-8')).decode('utf-8')
+    message = re.sub(r'<ds:SignatureValue>.*?</ds:SignatureValue>', f'<ds:SignatureValue>{fake_sig}</ds:SignatureValue>', message)
+    result = await client._perform_request('OadrPoll', message)
+    assert result == (None, {})
 
+    logs = [rec.message for rec in caplog.records]
+    for log in logs:
+        if log.startswith("Non-OK status 403 when performing a request"):
+          assert "Invalid Signature" in log
+          break
+    else:
+        assert False
+
+def problematic_handler(*args, **kwargs):
+    raise Exception("BOOM")
 
 @pytest.mark.asyncio
-async def test_signature_error(start_server_with_signatures):
-    client = OpenADRClient(vtn_url=f"http://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b", ven_name=VEN_NAME,
-                           vtn_fingerprint="INVALID")
-    client.on_event = _client_on_event
+async def test_server_handler_exception(caplog):
+    server = OpenADRServer(vtn_id=VTN_ID,
+                           http_port=SERVER_PORT)
+    server.add_handler('on_create_party_registration', problematic_handler)
+    await server.run_async()
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url=f'http://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b')
+    await client.run()
+    #await asyncio.sleep(0.5)
+    await client.stop()
+    await server.stop()
+    for message in caplog.messages:
+        if message.startswith('Non-OK status 500 when performing a request'):
+            break
+    else:
+        assert False
+
+def protocol_error_handler(*args, **kwargs):
+    raise errors.OutOfSequenceError()
+
+
+@pytest.mark.asyncio
+async def test_throw_protocol_error(caplog):
+    server = OpenADRServer(vtn_id=VTN_ID,
+                           http_port=SERVER_PORT)
+    server.add_handler('on_create_party_registration', protocol_error_handler)
+    await server.run_async()
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url=f'http://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b')
     await client.run()
-    await asyncio.sleep(3)
+    #await asyncio.sleep(0.5)
+    await client.stop()
+    await server.stop()
+    assert 'We got a non-OK OpenADR response from the server: 450: OUT OF SEQUENCE' in caplog.messages
+
+@pytest.mark.asyncio
+async def test_invalid_signature_error(start_server_with_signatures, caplog):
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url=f'https://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b',
+                           cert=VEN_CERT,
+                           key=VEN_KEY,
+                           vtn_fingerprint=VTN_FINGERPRINT)
+    message = client._create_message('oadrPoll', ven_id='ven123')
+    fake_sig = b64encode("HelloThere".encode('utf-8')).decode('utf-8')
+    message = re.sub(r'<ds:SignatureValue>.*?</ds:SignatureValue>', f'<ds:SignatureValue>{fake_sig}</ds:SignatureValue>', message)
+    result = await client._perform_request('OadrPoll', message)
+    assert result == (None, {})
+
+    logs = [rec.message for rec in caplog.records]
+    for log in logs:
+        if log.startswith("Non-OK status 403 when performing a request"):
+          assert "Invalid Signature" in log
+          break
+    else:
+        assert False
+
+def test_replay_protect_message_too_old(caplog):
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url=f'https://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b',
+                           cert=VEN_CERT,
+                           key=VEN_KEY,
+                           vtn_fingerprint=VTN_FINGERPRINT)
+    _temp = messaging.REPLAY_PROTECT_MAX_TIME_DELTA
+    messaging.REPLAY_PROTECT_MAX_TIME_DELTA = timedelta(seconds=0)
+    message = client._create_message('oadrPoll', ven_id='ven123')
+    tree = etree.fromstring(message.encode('utf-8'))
+    with pytest.raises(ValueError) as err:
+        messaging._verify_replay_protect(tree)
+    assert str(err.value) == 'The message was signed too long ago.'
+    messaging.REPLAY_PROTECT_MAX_TIME_DELTA = _temp
+
+def test_replay_protect_repeated_message(caplog):
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url=f'https://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b',
+                           cert=VEN_CERT,
+                           key=VEN_KEY,
+                           vtn_fingerprint=VTN_FINGERPRINT)
+    message = client._create_message('oadrPoll', ven_id='ven123')
+    tree = etree.fromstring(message.encode('utf-8'))
+    messaging._verify_replay_protect(tree)
+    with pytest.raises(ValueError) as err:
+        messaging._verify_replay_protect(tree)
+    assert str(err.value) == 'This combination of timestamp and nonce was already used.'
+
+
+def test_replay_protect_missing_nonce(caplog):
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url=f'https://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b',
+                           cert=VEN_CERT,
+                           key=VEN_KEY,
+                           vtn_fingerprint=VTN_FINGERPRINT)
+    message = client._create_message('oadrPoll', ven_id='ven123')
+    message = re.sub('<dsp:nonce>.*?</dsp:nonce>', '', message)
+    tree = etree.fromstring(message.encode('utf-8'))
+    with pytest.raises(ValueError) as err:
+        messaging._verify_replay_protect(tree)
+    assert str(err.value) == "Missing 'nonce' element in ReplayProtect in incoming message."
+
+
+def test_replay_protect_malformed_nonce(caplog):
+    client = OpenADRClient(ven_name='myven',
+                           vtn_url=f'https://localhost:{SERVER_PORT}/OpenADR2/Simple/2.0b',
+                           cert=VEN_CERT,
+                           key=VEN_KEY,
+                           vtn_fingerprint=VTN_FINGERPRINT)
+    message = client._create_message('oadrPoll', ven_id='ven123')
+    message = re.sub('<dsp:timestamp>.*?</dsp:timestamp>', '', message)
+    tree = etree.fromstring(message.encode('utf-8'))
+    with pytest.raises(ValueError) as err:
+        messaging._verify_replay_protect(tree)
+    assert str(err.value) == "Missing or malformed ReplayProtect element in the message signature."
+
+    message = re.sub('<dsp:ReplayProtect>.*?</dsp:ReplayProtect>', '', message)
+    tree = etree.fromstring(message.encode('utf-8'))
+    with pytest.raises(ValueError) as err:
+        messaging._verify_replay_protect(tree)
+    assert str(err.value) == "Missing or malformed ReplayProtect element in the message signature."
+
+
+def test_server_add_unknown_handler(caplog):
+    server = OpenADRServer(vtn_id='myvtn')
+    with pytest.raises(NameError) as err:
+        server.add_handler('unknown_name', print)
+    assert str(err.value) == ("Unknown handler 'unknown_name'. Correct handler names are: "
+                              "'on_created_event', 'on_request_event', 'on_register_report', "
+                              "'on_create_report', 'on_created_report', 'on_request_report', "
+                              "'on_update_report', 'on_poll', 'on_query_registration', "
+                              "'on_create_party_registration', 'on_cancel_party_registration'.")
+
 
 ##########################################################################################
 
@@ -34,9 +223,17 @@ VEN_NAME = 'myven'
 VEN_ID = '1234abcd'
 VTN_ID = "TestVTN"
 
-CERTFILE = os.path.join(os.path.dirname(__file__), "cert.pem")
-KEYFILE =  os.path.join(os.path.dirname(__file__), "key.pem")
+VEN_CERT = os.path.join(os.path.dirname(os.path.dirname(__file__)), "certificates", "dummy_ven.crt")
+VEN_KEY = os.path.join(os.path.dirname(os.path.dirname(__file__)), "certificates", "dummy_ven.key")
+VTN_CERT = os.path.join(os.path.dirname(os.path.dirname(__file__)), "certificates", "dummy_vtn.crt")
+VTN_KEY = os.path.join(os.path.dirname(os.path.dirname(__file__)), "certificates", "dummy_vtn.key")
+CA_FILE = os.path.join(os.path.dirname(os.path.dirname(__file__)), "certificates", "dummy_ca.crt")
 
+with open(VEN_CERT) as file:
+    VEN_FINGERPRINT = certificate_fingerprint(file.read())
+
+with open(VTN_CERT) as file:
+    VTN_FINGERPRINT = certificate_fingerprint(file.read())
 
 async def _on_create_party_registration(payload):
     registration_id = generate_id()
@@ -56,27 +253,29 @@ async def _client_on_event(event):
 async def _client_on_report(report):
     pass
 
+def fingerprint_lookup(ven_id):
+    return VEN_FINGERPRINT
+
 @pytest.fixture
 async def start_server():
-    server = OpenADRServer(vtn_id=VTN_ID)
+    server = OpenADRServer(vtn_id=VTN_ID, http_port=SERVER_PORT)
     server.add_handler('on_create_party_registration', _on_create_party_registration)
-
-    runner = web.AppRunner(server.app)
-    await runner.setup()
-    site = web.TCPSite(runner, 'localhost', SERVER_PORT)
-    await site.start()
+    await server.run_async()
     yield
-    await runner.cleanup()
+    await server.stop()
 
 @pytest.fixture
 async def start_server_with_signatures():
-    server = OpenADRServer(vtn_id=VTN_ID, cert=CERTFILE, key=KEYFILE, passphrase='openadr')
+    server = OpenADRServer(vtn_id=VTN_ID,
+                           cert=VTN_CERT,
+                           key=VTN_KEY,
+                           http_cert=VTN_CERT,
+                           http_key=VTN_KEY,
+                           http_ca_file=CA_FILE,
+                           http_port=SERVER_PORT,
+                           fingerprint_lookup=fingerprint_lookup)
     server.add_handler('on_create_party_registration', _on_create_party_registration)
 
-    runner = web.AppRunner(server.app)
-    await runner.setup()
-    site = web.TCPSite(runner, 'localhost', SERVER_PORT)
-    await site.start()
+    await server.run_async()
     yield
-    await runner.cleanup()
-
+    await server.stop()

test/test_fingerprint_cmdline.py

@@ -0,0 +1,18 @@
+import subprocess
+from openleadr import utils
+import os
+import sys
+
+def test_fingerprint_cmdline():
+    cert_path = os.path.join('certificates', 'dummy_ven.crt')
+    with open(cert_path) as file:
+        cert_str = file.read()
+    fingerprint = utils.certificate_fingerprint(cert_str)
+
+    if sys.platform.startswith('linux') or sys.platform.startswith('darwin'):
+        executable = os.path.join(sys.prefix, 'bin', 'fingerprint')
+    elif sys.platform.startswith('win'):
+        executable = os.path.join(sys.prefix, 'Scripts', 'fingerprint.exe')
+    result = subprocess.run([executable, cert_path], stdout=subprocess.PIPE)
+
+    assert fingerprint == result.stdout.decode('utf-8').strip()

test/test_message_conversion.py

@@ -14,32 +14,21 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from openleadr.utils import generate_id
-from openleadr.messaging import create_message, parse_message
+from openleadr.utils import generate_id, group_targets_by_type
+from openleadr.messaging import create_message, parse_message, validate_xml_schema
 from openleadr import enums
 from pprint import pprint
 from termcolor import colored
 from datetime import datetime, timezone, timedelta
+import pytest
+from pprint import pprint, pformat
+from lxml import etree
+from dataclasses import asdict
+import re
 
 DATE_FORMAT = "%Y-%m-%dT%H:%M:%S.%fZ"
 
-def test_messages():
-    for message_type, data in testcases:
-        _test_message(message_type, **data)
 
-def _test_message(message_type, **data):
-    message = create_message(message_type, **data)
-    # print(message)
-    parsed = parse_message(message)[1]
-
-    if parsed == data:
-        print(colored(f"pass {message_type}", "green"))
-    else:
-        pprint(data)
-        print(message)
-        pprint(parsed)
-        print(colored(f"fail {message_type}", "red"))
-        quit(1)
 
 def create_dummy_event(ven_id):
     """
@@ -68,19 +57,48 @@ def create_dummy_event(ven_id):
                                     {"duration": timedelta(minutes=1), "uid": 7, "signal_payload": 10.0},
                                     {"duration": timedelta(minutes=1), "uid": 8, "signal_payload": 20.0}],
                     "signal_name": "LOAD_CONTROL",
-                    #"signal_name": "simple",
-                    #"signal_type": "level",
                     "signal_type": "x-loadControlCapacity",
                     "signal_id": generate_id(),
+                    "measurement": {"name": "voltage",
+                                    "description": "Voltage",
+                                    "unit": "V",
+                                    "scale": "none"},
                     "current_value": 0.0}]
     event_targets = [{"ven_id": 'VEN001'}, {"ven_id": 'VEN002'}]
     event = {'active_period': active_period,
              'event_descriptor': event_descriptor,
              'event_signals': event_signals,
              'targets': event_targets,
+             'targets_by_type': group_targets_by_type(event_targets),
              'response_required': 'always'}
     return event
 
+reports = [{'report_id': generate_id(),
+            'duration': timedelta(seconds=3600),
+            'report_descriptions': [{'r_id': generate_id(),
+                                     'report_subject': {'end_device_asset': {'mrid': 'meter001'}},
+                                     'report_data_source': {'resource_id': 'resource001'},
+                                     'report_type': 'usage',
+                                     'measurement': asdict(measurement),
+                                     'reading_type': 'Direct Read',
+                                     'market_context': 'http://MarketContext1',
+                                     'sampling_rate': {'min_period': timedelta(seconds=10), 'max_period': timedelta(seconds=30), 'on_change': False}} for measurement in enums.MEASUREMENTS.values],
+            'report_specifier_id': generate_id(),
+            'report_name': 'METADATA_HISTORY_USAGE',
+            'report_request_id': None,
+            'created_date_time': datetime.now(timezone.utc)}]
+
+for report in reports:
+  for rd in report['report_descriptions']:
+    rd['measurement'].pop('acceptable_units')
+    rd['measurement'].pop('ns')
+    if rd['measurement']['power_attributes'] is None:
+      rd['measurement'].pop('power_attributes')
+    if rd['measurement']['scale'] is None:
+      rd['measurement'].pop('scale')
+    if rd['measurement']['pulse_factor'] is None:
+      rd['measurement'].pop('pulse_factor')
+
 testcases = [
 ('oadrCanceledOpt', dict(response={'response_code': 200, 'response_description': 'OK', 'request_id': generate_id()}, opt_id=generate_id())),
 ('oadrCanceledPartyRegistration', dict(response={'response_code': 200, 'response_description': 'OK', 'request_id': generate_id()}, registration_id=generate_id(), ven_id='123ABC')),
@@ -120,6 +138,7 @@ testcases = [
                               event_id=generate_id(),
                               modification_number=1,
                               targets=[{'ven_id': '123ABC'}],
+                              targets_by_type=group_targets_by_type([{'ven_id': '123ABC'}]),
                               ven_id='VEN123')),
 ('oadrCreatePartyRegistration', dict(request_id=generate_id(), ven_id='123ABC', profile_name='2.0b', transport_name='simpleHttp', transport_address='http://localhost', report_only=False, xml_signature=False, ven_name='test', http_pull_model=True)),
 ('oadrCreateReport', dict(request_id=generate_id(),
@@ -128,12 +147,12 @@ testcases = [
                                                   'report_specifier': {'granularity': timedelta(seconds=900),
                                                                        'report_back_duration': timedelta(seconds=900),
                                                                        'report_interval': {'dtstart': datetime(2019, 11, 19, 11, 0, 18, 672768, tzinfo=timezone.utc),
-                                                                                           'duration': timedelta(seconds=7200),
-                                                                                           'tolerance': {'tolerate': {'startafter': timedelta(seconds=300)}}},
+                                                                                           'duration': timedelta(seconds=7200)},
                                                                        'report_specifier_id': '9c8bdc00e7',
-                                                                       'specifier_payload': {'r_id': 'd6e2e07485',
-                                                                                             'reading_type': 'Direct Read'}}}])),
+                                                                       'specifier_payloads': [{'r_id': 'd6e2e07485',
+                                                                                             'reading_type': 'Direct Read'}]}}])),
 ('oadrDistributeEvent', dict(request_id=generate_id(), response={'request_id': 123, 'response_code': 200, 'response_description': 'OK'}, events=[create_dummy_event(ven_id='123ABC')], vtn_id='VTN123')),
+('oadrDistributeEvent', dict(request_id=generate_id(), response={'request_id': 123, 'response_code': 200, 'response_description': 'OK'}, events=[create_dummy_event(ven_id='123ABC'), create_dummy_event(ven_id='123ABC')], vtn_id='VTN123')),
 ('oadrPoll', dict(ven_id='123ABC')),
 ('oadrQueryRegistration', dict(request_id=generate_id())),
 ('oadrRegisteredReport', dict(ven_id='VEN123', response={'response_code': 200, 'response_description': 'OK', 'request_id': generate_id()},
@@ -142,135 +161,164 @@ testcases = [
                                                                             'granularity': timedelta(minutes=15),
                                                                             'report_back_duration': timedelta(minutes=15),
                                                                             'report_interval': {'dtstart': datetime.now(timezone.utc),
-                                                                                                'duration': timedelta(hours=2),
-                                                                                                'tolerance': {'tolerate': {'startafter': timedelta(minutes=5)}},
-                                                                                                'notification': timedelta(minutes=30),
-                                                                                                'ramp_up': timedelta(minutes=15),
-                                                                                                'recovery': timedelta(minutes=5)},
-                                                                            'specifier_payload': {'r_id': generate_id(),
-                                                                                                  'reading_type': 'Direct Read'}}},
+                                                                                                'duration': timedelta(hours=2)},
+                                                                            'specifier_payloads': [{'r_id': generate_id(),
+                                                                                                    'reading_type': 'Direct Read'}]}},
                                                       {'report_request_id': generate_id(),
                                                        'report_specifier': {'report_specifier_id': generate_id(),
                                                                             'granularity': timedelta(minutes=15),
                                                                             'report_back_duration': timedelta(minutes=15),
                                                                             'report_interval': {'dtstart': datetime.now(timezone.utc),
-                                                                                                'duration': timedelta(hours=2),
-                                                                                                'tolerance': {'tolerate': {'startafter': timedelta(minutes=5)}},
-                                                                                                'notification': timedelta(minutes=30),
-                                                                                                'ramp_up': timedelta(minutes=15),
-                                                                                                'recovery': timedelta(minutes=5)},
-                                                                            'specifier_payload': {'r_id': generate_id(),
-                                                                                                  'reading_type': 'Direct Read'}}}])),
+                                                                                                'duration': timedelta(hours=2)},
+                                                                            'specifier_payloads': [{'r_id': generate_id(),
+                                                                                                    'reading_type': 'Direct Read'}]}}])),
 ('oadrRequestEvent', dict(request_id=generate_id(), ven_id='123ABC')),
 ('oadrRequestReregistration', dict(ven_id='123ABC')),
 ('oadrRegisterReport', dict(request_id=generate_id(), reports=[{'report_id': generate_id(),
-                                                                       'report_descriptions': {
-                                                                            generate_id(): {
-                                                                            'report_subjects': [{'ven_id': '123ABC'}],
-                                                                            'report_data_sources': [{'ven_id': '123ABC'}],
-                                                                            'report_type': 'reading',
-                                                                            'reading_type': 'Direct Read',
-                                                                            'market_context': 'http://localhost',
-                                                                            'sampling_rate': {'min_period': timedelta(minutes=1), 'max_period': timedelta(minutes=1), 'on_change': True}}},
-                                                                       'report_request_id': generate_id(),
-                                                                       'report_specifier_id': generate_id(),
-                                                                       'report_name': 'HISTORY_USAGE',
-                                                                       'created_date_time': datetime.now(timezone.utc)}],
+                                                                'report_descriptions': [{
+                                                                     'r_id': generate_id(),
+                                                                     'report_subject': {'end_device_asset': {'mrid': 'meter001'}},
+                                                                     'report_data_source': {'resource_id': '123ABC'},
+                                                                     'report_type': 'reading',
+                                                                     'reading_type': 'Direct Read',
+                                                                     'market_context': 'http://localhost',
+                                                                     'sampling_rate': {'min_period': timedelta(minutes=1), 'max_period': timedelta(minutes=1), 'on_change': True}}],
+                                                                'report_request_id': generate_id(),
+                                                                'report_specifier_id': generate_id(),
+                                                                'report_name': 'HISTORY_USAGE',
+                                                                'created_date_time': datetime.now(timezone.utc)}],
                                                         ven_id='123ABC',
                                                         report_request_id=generate_id())),
 ('oadrRegisterReport', {'request_id': '8a4f859883', 'reports': [{'report_id': generate_id(),
                                                                  'duration': timedelta(seconds=7200),
-                                                                 'report_descriptions': {'resource1_status': {
-                                                                                          'report_data_sources': [{'resource_id': 'resource1'}],
+                                                                 'report_descriptions': [{'r_id': generate_id(),
+                                                                                          'report_data_source': {'resource_id': 'resource1'},
                                                                                           'report_type': 'x-resourceStatus',
                                                                                           'reading_type': 'x-notApplicable',
                                                                                           'market_context': 'http://MarketContext1',
-                                                                                          'sampling_rate': {'min_period': timedelta(seconds=60), 'max_period': timedelta(seconds=60), 'on_change': False}}},
-                                                                  'report_request_id': '0',
+                                                                                          'sampling_rate': {'min_period': timedelta(seconds=60), 'max_period': timedelta(seconds=60), 'on_change': False}}],
+                                                                  'report_request_id': generate_id(),
                                                                   'report_specifier_id': '789ed6cd4e_telemetry_status',
                                                                   'report_name': 'METADATA_TELEMETRY_STATUS',
                                                                   'created_date_time': datetime(2019, 11, 20, 15, 4, 52, 638621, tzinfo=timezone.utc)},
                                                                  {'report_id': generate_id(),
                                                                   'duration': timedelta(seconds=7200),
-                                                                  'report_descriptions': {'resource1_energy': {
-                                                                                           'report_data_sources': [{'resource_id': 'resource1'}],
+                                                                  'report_descriptions': [{'r_id': 'resource1_energy',
+                                                                                           'report_data_source': {'resource_id': 'resource1'},
                                                                                            'report_type': 'usage',
-                                                                                           'energy_real': {'item_description': 'RealEnergy',
-                                                                                                           'item_units': 'Wh',
-                                                                                                           'si_scale_code': 'n'},
+                                                                                           'measurement': {'name': 'energyReal',
+                                                                                                           'description': 'RealEnergy',
+                                                                                                           'ns': 'power',
+                                                                                                           'unit': 'Wh',
+                                                                                                           'scale': 'n'},
                                                                                            'reading_type': 'Direct Read',
                                                                                            'market_context': 'http://MarketContext1',
                                                                                            'sampling_rate': {'min_period': timedelta(seconds=60), 'max_period': timedelta(seconds=60), 'on_change': False}},
-                                                                                          'resource1_power': {
-                                                                                           'report_data_sources': [{'resource_id': 'resource1'}],
+                                                                                          {'r_id': 'resource1_power',
+                                                                                           'report_data_source': {'resource_id': 'resource1'},
                                                                                            'report_type': 'usage',
-                                                                                           'power_real': {'item_description': 'RealPower',
-                                                                                                          'item_units': 'W',
-                                                                                                          'si_scale_code': 'n',
-                                                                                                          'power_attributes': {'hertz': 60, 'voltage': 110, 'ac': False}},
+                                                                                           'measurement': {'name': 'powerReal',
+                                                                                                           'description': 'RealPower',
+                                                                                                           'ns': 'power',
+                                                                                                           'unit': 'W',
+                                                                                                           'scale': 'n',
+                                                                                                           'power_attributes': {'hertz': 50, 'voltage': 230, 'ac': True}},
                                                                                             'reading_type': 'Direct Read',
                                                                                             'market_context': 'http://MarketContext1',
-                                                                                            'sampling_rate': {'min_period': timedelta(seconds=60), 'max_period': timedelta(seconds=60), 'on_change': False}}},
-                                                                  'report_request_id': '0',
+                                                                                            'sampling_rate': {'min_period': timedelta(seconds=60), 'max_period': timedelta(seconds=60), 'on_change': False}}],
+                                                                  'report_request_id': generate_id(),
                                                                   'report_specifier_id': '789ed6cd4e_telemetry_usage',
                                                                   'report_name': 'METADATA_TELEMETRY_USAGE',
                                                                   'created_date_time': datetime(2019, 11, 20, 15, 4, 52, 638621, tzinfo=timezone.utc)},
                                                                  {'report_id': generate_id(),
                                                                   'duration': timedelta(seconds=7200),
-                                                                  'report_descriptions': {'resource1_energy': {
-                                                                                           'report_data_sources': [{'resource_id': 'resource1'}],
+                                                                  'report_descriptions': [{'r_id': 'resource1_energy',
+                                                                                           'report_data_source': {'resource_id': 'resource1'},
                                                                                            'report_type': 'usage',
-                                                                                           'energy_real': {'item_description': 'RealEnergy',
-                                                                                                           'item_units': 'Wh',
-                                                                                                           'si_scale_code': 'n'},
+                                                                                           'measurement': {'name': 'energyReal',
+                                                                                                           'description': 'RealEnergy',
+                                                                                                           'ns': 'power',
+                                                                                                           'unit': 'Wh',
+                                                                                                           'scale': 'n'},
                                                                                            'reading_type': 'Direct Read',
                                                                                            'market_context': 'http://MarketContext1',
                                                                                            'sampling_rate': {'min_period': timedelta(seconds=60), 'max_period': timedelta(seconds=60), 'on_change': False}},
-                                                                                          'resource1_power': {
-                                                                                           'report_data_sources': [{'resource_id': 'resource1'}],
+                                                                                          {'r_id': 'resource1_power',
+                                                                                           'report_data_source': {'resource_id': 'resource1'},
                                                                                            'report_type': 'usage',
-                                                                                           'power_real': {'item_description': 'RealPower',
-                                                                                                          'item_units': 'W', 'si_scale_code': 'n',
-                                                                                                          'power_attributes': {'hertz': 60, 'voltage': 110, 'ac': False}},
-                                                                                           'reading_type': 'Direct Read',
-                                                                                           'market_context': 'http://MarketContext1',
-                                                                                           'sampling_rate': {'min_period': timedelta(seconds=60), 'max_period': timedelta(seconds=60), 'on_change': False}}},
-                                                                  'report_request_id': '0',
+                                                                                           'measurement': {'name': 'powerReal',
+                                                                                                           'description': 'RealPower',
+                                                                                                           'ns': 'power',
+                                                                                                           'unit': 'W',
+                                                                                                           'scale': 'n',
+                                                                                                           'power_attributes': {'hertz': 50, 'voltage': 230, 'ac': True}},
+                                                                                            'reading_type': 'Direct Read',
+                                                                                            'market_context': 'http://MarketContext1',
+                                                                                            'sampling_rate': {'min_period': timedelta(seconds=60), 'max_period': timedelta(seconds=60), 'on_change': False}}],
+                                                                  'report_request_id': generate_id(),
                                                                   'report_specifier_id': '789ed6cd4e_history_usage',
                                                                   'report_name': 'METADATA_HISTORY_USAGE',
                                                                   'created_date_time': datetime(2019, 11, 20, 15, 4, 52, 638621, tzinfo=timezone.utc)}], 'ven_id': 's3cc244ee6'}),
+('oadrRegisterReport', {'ven_id': 'ven123', 'request_id': generate_id(), 'reports': reports}),
 ('oadrResponse', dict(response={'response_code': 200, 'response_description': 'OK', 'request_id': generate_id()}, ven_id='123ABC')),
 ('oadrResponse', dict(response={'response_code': 200, 'response_description': 'OK', 'request_id': None}, ven_id='123ABC')),
 ('oadrUpdatedReport', dict(response={'response_code': 200, 'response_description': 'OK', 'request_id': generate_id()}, ven_id='123ABC', cancel_report={'request_id': generate_id(), 'report_request_id': [generate_id(), generate_id(), generate_id()], 'report_to_follow': False, 'ven_id': '123ABC'})),
 ('oadrUpdateReport', dict(request_id=generate_id(), reports=[{'report_id': generate_id(),
-                                                                                  'report_name': enums.REPORT_NAME.values[0],
-                                                                                  'created_date_time': datetime.now(timezone.utc),
-                                                                                  'report_request_id': generate_id(),
-                                                                                  'report_specifier_id': generate_id(),
-                                                                                  'report_descriptions': {generate_id(): {'report_subjects': [{'ven_id': '123ABC'}, {'ven_id': 'DEF456'}],
-                                                                                                                          'report_data_sources': [{'ven_id': '123ABC'}],
-                                                                                                                          'report_type': enums.REPORT_TYPE.values[0],
-                                                                                                                          'reading_type': enums.READING_TYPE.values[0],
-                                                                                                                          'market_context': 'http://localhost',
-                                                                                                                          'sampling_rate': {'min_period': timedelta(minutes=1),
-                                                                                                                                            'max_period': timedelta(minutes=2),
-                                                                                                                                            'on_change': False}}}}], ven_id='123ABC'))
-# for report_name in enums.REPORT_NAME.values:
-#     for reading_type in enums.READING_TYPE.values:
-#         for report_type in enums.REPORT_TYPE.values:
-#             ('oadrUpdateReport', dict(request_id=generate_id(), reports=[{'report_id': generate_id()),
-#                                                                                   'report_name': report_name,
-#                                                                                   'created_date_time': datetime.now(timezone.utc),
-#                                                                                   'report_request_id': generate_id(),
-#                                                                                   'report_specifier_id': generate_id(),
-#                                                                                   'report_descriptions': [{'r_id': generate_id(),
-#                                                                                                            'report_subjects': [{'ven_id': '123ABC'}, {'ven_id': 'DEF456'}],
-#                                                                                                            'report_data_sources': [{'ven_id': '123ABC'}],
-#                                                                                                            'report_type': report_type,
-#                                                                                                            'reading_type': reading_type,
-#                                                                                                            'market_context': 'http://localhost',
-#                                                                                                            'sampling_rate': {'min_period': timedelta(minutes=1),
-#                                                                                                                              'max_period': timedelta(minutes=2),
-#                                                                                                                              'on_change': False}}]}], ven_id='123ABC')
+                                                              'report_name': enums.REPORT_NAME.values[0],
+                                                              'created_date_time': datetime.now(timezone.utc),
+                                                              'report_request_id': generate_id(),
+                                                              'report_specifier_id': generate_id(),
+                                                              'report_descriptions': [{'r_id': generate_id(),
+                                                                                       'report_subject': {'end_device_asset': {'mrid': 'meter001'}},
+                                                                                       'report_data_source': {'resource_id': '123ABC'},
+                                                                                       'report_type': enums.REPORT_TYPE.values[0],
+                                                                                       'reading_type': enums.READING_TYPE.values[0],
+                                                                                       'market_context': 'http://localhost',
+                                                                                       'sampling_rate': {'min_period': timedelta(minutes=1),
+                                                                                                         'max_period': timedelta(minutes=2),
+                                                                                                         'on_change': False}}
+                                                                                        ]}], ven_id='123ABC'))
 
-]
+]
+
+@pytest.mark.parametrize('message_type,data', testcases)
+def test_message(message_type, data):
+    # file = open('representations.rst', 'a')
+    # print(f".. _{message_type}:", file=file)
+    # print("", file=file)
+    # print(message_type, file=file)
+    # print("="*len(message_type), file=file)
+    # print("", file=file)
+    # print("OpenADR payload:", file=file)
+    # print("", file=file)
+    # print(".. code-block:: xml", file=file)
+    # print("    ", file=file)
+    message = create_message(message_type, **data)
+    # message = re.sub(r"\s\s+","",message)
+    # message = message.replace("\n","")
+    # xml_lines = etree.tostring(etree.fromstring(message.replace('\n', '').encode('utf-8')), pretty_print=True).decode('utf-8').splitlines()
+    # for line in xml_lines:
+    #      print("    " + line, file=file)
+    # print("", file=file)
+    # print("OpenLEADR representation:", file=file)
+    # print("    ", file=file)
+    # print(".. code-block:: python3", file=file)
+    # print("    ", file=file)
+    validate_xml_schema(message)
+    parsed = parse_message(message)[1]
+    # dict_lines = pformat(parsed).splitlines()
+    # for line in dict_lines:
+    #     print("    " + line, file=file)
+    # print("", file=file)
+    # print("", file=file)
+    if message_type == 'oadrRegisterReport':
+        for report in data['reports']:
+            for rd in report['report_descriptions']:
+                if 'measurement' in rd:
+                    rd['measurement'].pop('ns')
+    if message_type == 'oadrDistributeEvent':
+        for event in data['events']:
+            for signal in event['event_signals']:
+                if 'measurement' in signal:
+                    signal['measurement'].pop('ns')
+    assert parsed == data

test/test_objects.py

@@ -1,7 +1,10 @@
 from openleadr import objects, enums
-from datetime import datetime, timedelta
-from openleadr.messaging import create_message, parse_message
+from datetime import datetime, timedelta, timezone
+from openleadr.utils import ensure_bytes
+from openleadr.messaging import create_message, parse_message, validate_xml_schema
 from pprint import pprint
+import pytest
+
 
 def test_oadr_event():
     event = objects.Event(
@@ -42,7 +45,185 @@ def test_oadr_event():
                                 response_description='OK',
                                 request_id='1234')
     msg = create_message('oadrDistributeEvent', response=response, events=[event])
+    validate_xml_schema(ensure_bytes(msg))
+    message_type, message_payload = parse_message(msg)
+
+
+def test_oadr_event_targets_by_type():
+    event = objects.Event(
+        event_descriptor=objects.EventDescriptor(
+            event_id=1,
+            modification_number=0,
+            market_context='MarketContext1',
+            event_status=enums.EVENT_STATUS.NEAR),
+        active_period=objects.ActivePeriod(
+            dtstart=datetime.now(),
+            duration=timedelta(minutes=10)),
+        event_signals=[objects.EventSignal(
+            intervals=[
+                objects.Interval(
+                    dtstart=datetime.now(),
+                    duration=timedelta(minutes=5),
+                    uid=0,
+                    signal_payload=1),
+                objects.Interval(
+                    dtstart=datetime.now(),
+                    duration=timedelta(minutes=5),
+                    uid=1,
+                    signal_payload=2)],
+            targets=[objects.Target(
+                ven_id='1234'
+            )],
+            signal_name=enums.SIGNAL_NAME.LOAD_CONTROL,
+            signal_type=enums.SIGNAL_TYPE.LEVEL,
+            signal_id=1,
+            current_value=0
+        )],
+        targets_by_type={'ven_id': ['ven123']}
+    )
+
+    msg = create_message('oadrDistributeEvent', events=[event])
+    validate_xml_schema(ensure_bytes(msg))
+    message_type, message_payload = parse_message(msg)
+
+
+def test_oadr_event_targets_and_targets_by_type():
+    event = objects.Event(
+        event_descriptor=objects.EventDescriptor(
+            event_id=1,
+            modification_number=0,
+            market_context='MarketContext1',
+            event_status=enums.EVENT_STATUS.NEAR),
+        active_period=objects.ActivePeriod(
+            dtstart=datetime.now(),
+            duration=timedelta(minutes=10)),
+        event_signals=[objects.EventSignal(
+            intervals=[
+                objects.Interval(
+                    dtstart=datetime.now(),
+                    duration=timedelta(minutes=5),
+                    uid=0,
+                    signal_payload=1),
+                objects.Interval(
+                    dtstart=datetime.now(),
+                    duration=timedelta(minutes=5),
+                    uid=1,
+                    signal_payload=2)],
+            targets=[objects.Target(
+                ven_id='1234'
+            )],
+            signal_name=enums.SIGNAL_NAME.LOAD_CONTROL,
+            signal_type=enums.SIGNAL_TYPE.LEVEL,
+            signal_id=1,
+            current_value=0
+        )],
+        targets=[{'ven_id': 'ven123'}],
+        targets_by_type={'ven_id': ['ven123']}
+    )
+
+    msg = create_message('oadrDistributeEvent', events=[event])
+    validate_xml_schema(ensure_bytes(msg))
     message_type, message_payload = parse_message(msg)
 
 
+def test_oadr_event_targets_and_targets_by_type_invalid():
+    with pytest.raises(ValueError):
+        event = objects.Event(
+            event_descriptor=objects.EventDescriptor(
+                event_id=1,
+                modification_number=0,
+                market_context='MarketContext1',
+                event_status=enums.EVENT_STATUS.NEAR),
+            active_period=objects.ActivePeriod(
+                dtstart=datetime.now(),
+                duration=timedelta(minutes=10)),
+            event_signals=[objects.EventSignal(
+                intervals=[
+                    objects.Interval(
+                        dtstart=datetime.now(),
+                        duration=timedelta(minutes=5),
+                        uid=0,
+                        signal_payload=1),
+                    objects.Interval(
+                        dtstart=datetime.now(),
+                        duration=timedelta(minutes=5),
+                        uid=1,
+                        signal_payload=2)],
+                targets=[objects.Target(
+                    ven_id='1234'
+                )],
+                signal_name=enums.SIGNAL_NAME.LOAD_CONTROL,
+                signal_type=enums.SIGNAL_TYPE.LEVEL,
+                signal_id=1,
+                current_value=0
+            )],
+            targets=[objects.Target(ven_id='ven456')],
+            targets_by_type={'ven_id': ['ven123']}
+        )
+
+        msg = create_message('oadrDistributeEvent', events=[event])
+        validate_xml_schema(ensure_bytes(msg))
+        message_type, message_payload = parse_message(msg)
+
+
+def test_oadr_event_no_targets():
+    with pytest.raises(ValueError):
+        event = objects.Event(
+            event_descriptor=objects.EventDescriptor(
+                event_id=1,
+                modification_number=0,
+                market_context='MarketContext1',
+                event_status=enums.EVENT_STATUS.NEAR),
+            active_period=objects.ActivePeriod(
+                dtstart=datetime.now(),
+                duration=timedelta(minutes=10)),
+            event_signals=[objects.EventSignal(
+                intervals=[
+                    objects.Interval(
+                        dtstart=datetime.now(),
+                        duration=timedelta(minutes=5),
+                        uid=0,
+                        signal_payload=1),
+                    objects.Interval(
+                        dtstart=datetime.now(),
+                        duration=timedelta(minutes=5),
+                        uid=1,
+                        signal_payload=2)],
+                targets=[objects.Target(
+                    ven_id='1234'
+                )],
+                signal_name=enums.SIGNAL_NAME.LOAD_CONTROL,
+                signal_type=enums.SIGNAL_TYPE.LEVEL,
+                signal_id=1,
+                current_value=0
+            )]
+        )
+
+def test_event_signal_with_grouped_targets():
+    event_signal = objects.EventSignal(intervals=[objects.Interval(dtstart=datetime.now(timezone.utc),
+                                                                  duration=timedelta(minutes=10),
+                                                                  signal_payload=1)],
+                                       signal_name='simple',
+                                       signal_type='level',
+                                       signal_id='signal123',
+                                       targets_by_type={'ven_id': ['ven123', 'ven456']})
+    assert event_signal.targets == [objects.Target(ven_id='ven123'), objects.Target(ven_id='ven456')]
+
+def test_event_signal_with_incongruent_targets():
+    with pytest.raises(ValueError):
+        event_signal = objects.EventSignal(intervals=[objects.Interval(dtstart=datetime.now(timezone.utc),
+                                                                      duration=timedelta(minutes=10),
+                                                                      signal_payload=1)],
+                                           signal_name='simple',
+                                           signal_type='level',
+                                           signal_id='signal123',
+                                           targets=[objects.Target(ven_id='ven123')],
+                                           targets_by_type={'ven_id': ['ven123', 'ven456']})
+
 
+def test_event_descriptor_modification_number():
+    event_descriptor = objects.EventDescriptor(event_id='event123',
+                                               modification_number=None,
+                                               market_context='http://marketcontext01',
+                                               event_status='near')
+    assert event_descriptor.modification_number == 0