udmi

UDMI / Docs / Specs / Sequences / Generated

Generated sequences

These are the exact sequences being checked by the sequence tool. They are programmatically generated so maybe a bit cryptic, but they accurately represent the specific steps required for each test.

bad_point_ref (PREVIEW)

Error handling for badly formed gateway point ref

Test skipped: Not a proxied device

bad_target_address (PREVIEW)

Error handling for badly formed gateway target address

Test skipped: Not a proxied device

bad_target_family (PREVIEW)

Error handling for badly formed gateway target family

Test skipped: Not a proxied device

broken_config (STABLE)

Check that the device correctly handles a broken (non-json) config message.

  1. Update config to enable debug logging
    • Set system.min_loglevel = 100
  2. logging Wait until system logs level NOTICE category system.config.apply
  3. Update config to force broken (invalid JSON) configuration
  4. status Wait until system status level is >= WARNING (400)
  5. status Check that status level is exactly ERROR (500)
  6. status Check that category matches system.config.parse
  7. Check that device state last_config has not been updated
  8. logging Wait until system logs level DEBUG category system.config.receive
  9. logging Wait until system logs level ERROR category system.config.parse
  10. logging Check that log level NOTICE (or greater) category system.config.apply was not logged
  11. Reset config to clean version
  12. (Log level is implicitly set to INFO through config reset)
  13. status Wait until system status level is not >= WARNING (400)
  14. logging Wait until system logs level NOTICE category system.config.apply
  15. logging Check that log level DEBUG (or greater) category system.config.receive was not logged
  16. logging Check that log level DEBUG (or greater) category system.config.parse was not logged
  17. Check that device state last_config has been updated

Test passed.

config_logging (STABLE)

Check that the device publishes minimum required log entries when receiving config

  1. Force config update to resend config to device
  2. Wait until system logs level DEBUG category system.config.receive
  3. Wait until system logs level DEBUG category system.config.parse
  4. Wait until system logs level NOTICE category system.config.apply

Test passed.

device_config_acked (STABLE)

Check that the device MQTT-acknowledges a sent config.

  1. Wait for config acked

Test passed.

endpoint_connection_error (PREVIEW)

Push endpoint config message to device that results in a connection error.

  1. Update config before blobset entry config status is error
    • Add blobset = { “blobs”: { “_iot_endpoint_config”: { “phase”: final, “generation”: blob generation, “sha256”: blob data hash, “url”: endpoint data } } }
  2. Wait for blobset entry config status is error
  3. Update config before endpoint config blobset state not defined
    • Remove blobset.blobs._iot_endpoint_config
  4. Wait for endpoint config blobset state not defined

Test passed.

endpoint_connection_retry (PREVIEW)

Check repeated endpoint with same information gets retried.

  1. Update config before blobset entry config status is error
    • Add blobset = { “blobs”: { “_iot_endpoint_config”: { “phase”: final, “generation”: blob generation, “sha256”: blob data hash, “url”: endpoint data } } }
  2. Wait for blobset entry config status is error
  3. Update config before blobset entry config status is error
    • Set blobset.blobs._iot_endpoint_config.generation = new generation
  4. Wait for blobset entry config status is error
  5. Update config before endpoint config blobset state not defined
    • Remove blobset.blobs._iot_endpoint_config
  6. Wait for endpoint config blobset state not defined

Test passed.

endpoint_connection_success_alternate (PREVIEW)

Check connection to an alternate project.

  1. Wait until initial last_config matches config timestamp
  2. Update config mirroring config false
    • Add blobset = { “blobs”: { “_iot_endpoint_config”: { “phase”: final, “generation”: blob generation, “sha256”: blob data hash, “url”: endpoint data } } }
  3. Wait for blobset phase is apply and stateStatus is null
  4. Wait for blobset phase is final and stateStatus is null
  5. Wait until alternate last_config matches config timestamp
  6. Update config before endpoint config blobset state not defined
    • Remove blobset.blobs._iot_endpoint_config
  7. Wait for endpoint config blobset state not defined
  8. Update config mirroring config true
    • Add blobset.blobs._iot_endpoint_config = { “phase”: final, “generation”: blob generation, “sha256”: blob data hash, “url”: endpoint data }
  9. Wait for blobset phase is apply and stateStatus is null
  10. Wait for blobset phase is final and stateStatus is null
  11. Wait until restored last_config matches config timestamp
  12. Update config before endpoint config blobset state not defined
    • Remove blobset.blobs._iot_endpoint_config
  13. Wait for endpoint config blobset state not defined

Test passed.

endpoint_connection_success_reconnect (PREVIEW)

Check a successful reconnect to the same endpoint.

  1. Update config before blobset phase is final and stateStatus is null
    • Add blobset = { “blobs”: { “_iot_endpoint_config”: { “phase”: final, “generation”: blob generation, “sha256”: blob data hash, “url”: endpoint data } } }
  2. Wait for blobset phase is final and stateStatus is null
  3. Update config before endpoint config blobset state not defined
    • Remove blobset.blobs._iot_endpoint_config
  4. Wait for endpoint config blobset state not defined

Test passed.

endpoint_failure_and_restart (PREVIEW)

  1. Update config before blobset entry config status is error
    • Add blobset = { “blobs”: { “_iot_endpoint_config”: { “phase”: final, “generation”: blob generation, “sha256”: blob data hash, “url”: endpoint data } } }
  2. Wait for blobset entry config status is error
  3. Wait for last_start is not zero
  4. Check that initial count is greater than 0
  5. Update config before system mode is ACTIVE
    • Add system.operation.mode = active
  6. Wait for system mode is ACTIVE
  7. Update config before system mode is INITIAL
    • Set system.operation.mode = restart
  8. Wait for system mode is INITIAL
  9. Check that restart count increased by one
  10. Update config before system mode is ACTIVE
    • Set system.operation.mode = active
  11. Wait for system mode is ACTIVE
  12. Wait for last_config is newer than previous last_config before abort
  13. Wait for last_config is newer than previous last_config after abort
  14. Wait for last_start is newer than previous last_start
  15. Update config before endpoint config blobset state not defined
    • Remove blobset.blobs._iot_endpoint_config
  16. Wait for endpoint config blobset state not defined

Test passed.

endpoint_redirect_and_restart (PREVIEW)

  1. Wait until initial last_config matches config timestamp
  2. Update config mirroring config false
    • Add blobset = { “blobs”: { “_iot_endpoint_config”: { “phase”: final, “generation”: blob generation, “sha256”: blob data hash, “url”: endpoint data } } }
  3. Wait for blobset phase is apply and stateStatus is null
  4. Wait for blobset phase is final and stateStatus is null
  5. Wait until alternate last_config matches config timestamp
  6. Update config before endpoint config blobset state not defined
    • Remove blobset.blobs._iot_endpoint_config
  7. Wait for endpoint config blobset state not defined
  8. Wait for last_start is not zero
  9. Check that initial count is greater than 0
  10. Update config before system mode is ACTIVE
    • Add system.operation.mode = active
  11. Wait for system mode is ACTIVE
  12. Update config before system mode is INITIAL
    • Set system.operation.mode = restart
  13. Wait for system mode is INITIAL
  14. Check that restart count increased by one
  15. Update config before system mode is ACTIVE
    • Set system.operation.mode = active
  16. Wait for system mode is ACTIVE
  17. Wait for last_config is newer than previous last_config before abort
  18. Wait for last_config is newer than previous last_config after abort
  19. Wait for last_start is newer than previous last_start
  20. Update config mirroring config true
    • Add blobset.blobs._iot_endpoint_config = { “phase”: final, “generation”: blob generation, “sha256”: blob data hash, “url”: endpoint data }
  21. Wait for blobset phase is apply and stateStatus is null
  22. Wait for blobset phase is final and stateStatus is null
  23. Wait until restored last_config matches config timestamp
  24. Update config before endpoint config blobset state not defined
    • Remove blobset.blobs._iot_endpoint_config
  25. Wait for endpoint config blobset state not defined

Test passed.

enumerate_families (PREVIEW)

Check enumeration of network families

  1. Update config before enumeration not active
    • Add discovery.enumerations = { “families”: entries }
  2. Wait for enumeration not active
  3. Update config before matching enumeration generation
    • Add discovery.generation = generation start time
  4. Wait for matching enumeration generation
  5. Update config before cleared enumeration generation
    • Remove discovery.generation
  6. Wait for cleared enumeration generation
  7. Check that family enumeration size matches
  8. Check that no feature enumeration exists
  9. Check that no point enumeration exists

Test passed.

enumerate_features (PREVIEW)

Check enumeration of device features

  1. Update config before enumeration not active
    • Add discovery.enumerations = { “features”: entries }
  2. Wait for enumeration not active
  3. Update config before matching enumeration generation
    • Add discovery.generation = generation start time
  4. Wait for matching enumeration generation
  5. Update config before cleared enumeration generation
    • Remove discovery.generation
  6. Wait for cleared enumeration generation
  7. Check that no family enumeration exists
  8. Check that feature enumeration matches metadata
  9. Check that all enumerated features are official buckets
  10. Check that no point enumeration exists

Test passed.

enumerate_multi (PREVIEW)

Check enumeration of multiple categories

  1. Update config before enumeration not active
    • Add discovery.enumerations = { “features”: entries, “families”: entries, “points”: entries }
  2. Wait for enumeration not active
  3. Update config before matching enumeration generation
    • Add discovery.generation = generation start time
  4. Wait for matching enumeration generation
  5. Update config before cleared enumeration generation
    • Remove discovery.generation
  6. Wait for cleared enumeration generation
  7. Check that family enumeration size matches
  8. Check that feature enumeration matches metadata
  9. Check that all enumerated features are official buckets
  10. Check that enumerated point count matches

Test passed.

enumerate_nothing (PREVIEW)

Check enumeration of nothing at all

  1. Update config before enumeration not active
    • Add discovery.enumerations = { }
  2. Wait for enumeration not active
  3. Update config before matching enumeration generation
    • Add discovery.generation = generation start time
  4. Wait for matching enumeration generation
  5. Update config before cleared enumeration generation
    • Remove discovery.generation
  6. Wait for cleared enumeration generation
  7. Check that no family enumeration exists
  8. Check that no feature enumeration exists
  9. Check that no point enumeration exists

Test passed.

enumerate_pointset (PREVIEW)

Check enumeration of device points

  1. Update config before enumeration not active
    • Add discovery.enumerations = { “points”: entries }
  2. Wait for enumeration not active
  3. Update config before matching enumeration generation
    • Add discovery.generation = generation start time
  4. Wait for matching enumeration generation
  5. Update config before cleared enumeration generation
    • Remove discovery.generation
  6. Wait for cleared enumeration generation
  7. Check that no family enumeration exists
  8. Check that no feature enumeration exists
  9. Check that enumerated point count matches

Test passed.

extra_config (STABLE)

Check that the device correctly handles an extra out-of-schema field

  1. Wait for last_config not null
  2. Wait for system operational
  3. Check that system status level is not >= WARNING (400)
  4. Update config before system logs level DEBUG category system.config.receive
  5. Wait until system logs level DEBUG category system.config.receive
  6. Wait until last_config updated
  7. Wait for system operational
  8. Check that system status level is not >= WARNING (400)
  9. Wait until system logs level DEBUG category system.config.parse
  10. Wait until system logs level NOTICE category system.config.apply
  11. Update config before system logs level DEBUG category system.config.receive
  12. Wait until system logs level DEBUG category system.config.receive
  13. Wait for last_config updated again
  14. Wait for system operational
  15. Wait until system logs level DEBUG category system.config.parse
  16. Wait until system logs level NOTICE category system.config.apply

Test passed.

family_ether_addr (PREVIEW)

  1. Wait until localnet family state ether available
  2. Check that family ether address matches

Test passed.

family_ipv4_addr (PREVIEW)

  1. Wait until localnet family state ipv4 available
  2. Check that family ipv4 address matches

Test passed.

family_ipv6_addr (PREVIEW)

Test skipped: No ipv6 address defined in metadata

gateway_proxy_events (BETA)

Check that a gateway proxies pointset events for indicated devices

Test skipped: Not a gateway

gateway_proxy_state (PREVIEW)

Check that a gateway proxies state updates for indicated devices

Test skipped: Not a gateway

pointset_publish (STABLE)

Check that a device publishes pointset events

  1. Wait for receive a pointset event

Test passed.

pointset_publish_interval (STABLE)

Check handling of sample_rate_sec and sample_limit_sec

  1. Update config before receive at least 4 pointset events
    • Add pointset.sample_rate_sec = 8
    • Add pointset.sample_limit_sec = 5
  2. Wait for receive at least 4 pointset events
  3. Check that time period between successive pointset events is between 5 and 8 seconds
  4. Update config before receive at least 4 pointset events
    • Set pointset.sample_rate_sec = 18
    • Set pointset.sample_limit_sec = 15
  5. Wait for receive at least 4 pointset events
  6. Check that time period between successive pointset events is between 15 and 18 seconds

Test passed.

pointset_remove_point (STABLE)

Check that pointset state does not report an unconfigured point

  1. Wait until pointset state matches config
  2. Wait until pointset event contains correct points
  3. Update config before pointset state does not contain removed point
    • Remove pointset.points[random_point]
  4. Wait for pointset state does not contain removed point
  5. Wait until pointset state matches config
  6. Wait until pointset event contains correct points
  7. Update config before pointset state contains restored point
    • Add pointset.points[random_point] = point configuration
  8. Wait for pointset state contains restored point
  9. Wait until pointset state matches config
  10. Wait until pointset event contains correct points

Test passed.

pointset_request_extraneous (STABLE)

Check error when pointset configuration contains extraneous point

  1. Update config before pointset state matches config
    • Add pointset.sample_rate_sec = 10
  2. Wait until pointset state matches config
  3. Wait until pointset event contains correct points
  4. Update config before pointset state contains extraneous point error
    • Add pointset.points[extraneous_point] = point configuration
  5. Wait for pointset state contains extraneous point error
  6. Wait until pointset state matches config
  7. Wait until pointset event contains correct points
  8. Update config before pointset state removes extraneous point error
    • Remove pointset.points[extraneous_point]
  9. Wait for pointset state removes extraneous point error
  10. Wait until pointset state matches config
  11. Wait until pointset event contains correct points

Test passed.

scan_periodic_now_enumerate (PREVIEW)

Check periodic scan on a fixed schedule and enumeration

  1. Update config before discovery families defined
    • Remove discovery.families.vendor
  2. Wait for discovery families defined
  3. Wait until discovery family keys match
  4. Wait for no scans active
  5. Update config before scan iterations
    • Add discovery.families.vendor = { “generation”: family generation, “scan_interval_sec”: 20, “depth”: entries, “scan_duration_sec”: 20 }
  6. Wait for scan iterations
  7. Check that scan did not terminate prematurely
  8. Check that all events have matching refs

Test passed.

scan_single_future (PREVIEW)

Check results of a single scan scheduled soon

  1. Update config before discovery families defined
    • Remove discovery.families.vendor
  2. Wait for discovery families defined
  3. Wait until discovery family keys match
  4. Wait for no scans active
  5. Update config before scheduled scan pending
    • Add discovery.families.vendor = { “generation”: family generation, “scan_duration_sec”: 10 }
  6. Wait until scheduled scan pending
  7. Wait until scheduled scan active
  8. Check that scan started at time
  9. Wait until scheduled scan complete
  10. Check that scan completed at time
  11. Check that discovery events were received
  12. Check that no events have discovered refs
  13. Check that discovery events were valid
  14. Check that all scan addresses are unique
  15. Check that all expected addresses were found

Test passed.

scan_single_now (PREVIEW)

Check results of a single scan scheduled in the recent past

  1. Update config before discovery families defined
    • Remove discovery.families.vendor
  2. Wait for discovery families defined
  3. Wait until discovery family keys match
  4. Wait for no scans active
  5. Update config before scheduled scan active
    • Add discovery.families.vendor = { “generation”: family generation, “scan_duration_sec”: 10 }
  6. Wait until scheduled scan active
  7. Check that scan started at time
  8. Wait until scheduled scan complete
  9. Check that scan completed at time
  10. Check that discovery events were received
  11. Check that no events have discovered refs
  12. Check that discovery events were valid
  13. Check that all scan addresses are unique
  14. Check that all expected addresses were found

Test passed.

scan_single_past (PREVIEW)

Check that a scan scheduled in the past never starts

  1. Update config before discovery families defined
    • Remove discovery.families.vendor
  2. Wait for discovery families defined
  3. Wait until discovery family keys match
  4. Wait for no scans active
  5. Update config before scan schedule initially not active
    • Add discovery.families.vendor = { “generation”: family generation, “scan_duration_sec”: 10 }
  6. Wait until scan schedule initially not active
  7. Wait until scan schedule still not active
  8. Check that there were no received discovery events

Test passed.

state_make_model (STABLE)

Check that a device publishes correct make and model information in state messages

  1. Check that make and model in state matches make in metadata

Test passed.

state_software (STABLE)

Check that a device publishes correct software information in state messages

  1. Check that software in metadata matches state

Test passed.

system_last_update (STABLE)

Check that last_update state is correctly set in response to a config update.

  1. Wait until state last_config matches config timestamp
  2. subblocks Wait until state update complete
  3. Force config update to trigger another config update
  4. Wait until state last_config matches config timestamp
  5. subblocks Wait until state update complete
  6. Force config update to trigger another config update
  7. Wait until state last_config matches config timestamp
  8. subblocks Wait until state update complete

Test passed.

valid_serial_no (STABLE)

  1. Wait for received serial number matches

Test passed.

This site is open source. Improve this page.