Skip to main content

Decipher Support

All the topics, resources needed for Decipher.


FocusVision Knowledge Base

Validating Hidden/Virtual Questions

  Requires Decipher Cloud

1:  About Validating Hidden/Virtual Questions

You can validate hidden/virtual questions to verify that the data is being populated. To do this, first create a text file with the rules of assertions and conditions.  Next save this file to the project directory and then run the validation script.  The test results are returned. 

For any hidden/virtual question that is not being populated as expected, you can alert the appropriate person of the situation. Validating hidden/virtual questions adds an extra level of reliability to a project.

2: Setting Up the Rules File

Create a text file that contains the rules for validation.

assert variable=value
requires any/all [var1=val1] ... [varN=valN]
must test variable

The first line is the assertion statement which can only check one variable.  There must be a space after assert since the syntax is space delimited.  The second line specifies the conditions that must be met for the assertion above to be true (requires any / requires all) and can have many conditions (delimited by spaces).  Note that:

  • any joins the conditions being checked with or . Any of the conditions can be true for the test to pass. 
  • all joins the vars with and. All of the conditions must be true for the test to pass. 

An assertion statement must be followed by a conditions statement.  The third line is the list of variables which require at least one test (one assertion plus conditions) for the validation to complete as successful.  The must test statement is optional and can be placed anywhere in the file.

2.1:  Supported Statements

The following provides additional details of the supported statements:

  • Recognized statements include assert...requires any/all and must test
  • Variables/values are from the default tab-delimited data file, e.g.:
    • Variables: q1, q2r1c1, etc.
      • If you renamed anything with altlabels, that will change the variable names; use the default datamap to determine the variable names
    • Values:
      • Checkbox: 0 or 1
      • Radio/Select: 1, 2, 3, …
      • Text/Textarea/Number: number/text (spaces and ranges are not supported)
    • To test for an empty/null value, leave the right side of the expression blank (e.g., "q1=")
    • Equalities are space-delimited, so you must not have spaces within them (e.g., you cannot have q1 = 3 or qName=Ronald McDonald or alt label=1)
  • * (asterisk character) can be used as a wildcard for matching the variable for an assert to the variable/value for a condition

2.2:  Examples of the assert Function

The following tables in example 1 and 2 show the xml of a virtual question and the corresponding assert function that may be written to test it. Example 3 demonstrates the use of wildcards in assertions.

Example 1

Virtual Question Assertion
 <radio optional="0" label="q1">
    <row label="r1">Male</row>
    <row label="r2">Female</row>
    <comment>Select one</comment>
    <title>Are you...?</title>
  <radio label="q2">
    <title>What is your age?</title>
    <row label="r1">Under 18</row>
    <row label="r2">18-24</row>
    <row label="r3">25-34</row>
    <row label="r4">35-44</row>
    <row label="r5">45-54</row>
    <row label="r6">55+</row>
  <radio label="AGEGEN" title="AGE / GENDER">
if not q2.r1:
    if q1.r1:
        data.val = q2.val - 1
        data.val = data.r6.index + (q2.val - 1)
    <row label="r1">Male, 18-24</row>
    <row label="r2">Male, 25-34</row>
    <row label="r3">Male, 35-44</row>
    <row label="r4">Male, 45-54</row>
    <row label="r5">Male, 55+</row>
    <row label="r6">Female, 18-24</row>
    <row label="r7">Female, 25-34</row>
    <row label="r8">Female, 35-44</row>
    <row label="r9">Female, 45-54</row>
    <row label="r10">Female, 55+</row>
assert AGEGEN=1
# TODO: add more tests
requires all q1=1 q2=2

assert AGEGEN=2
requires all q1=1 q2=3

assert AGEGEN=3
requires all q1=1 q2=4

assert AGEGEN=4
requires all q1=1 q2=5

assert AGEGEN=5
requires all q1=1 q2=6

assert AGEGEN=6
requires all q1=2 q2=2

assert AGEGEN=7
requires all q1=2 q2=3

assert AGEGEN=8
requires all q1=2 q2=4

assert AGEGEN=9
requires all q1=2 q2=5

assert AGEGEN=10
requires all q1=2 q2=6

Example 2

Virtual Question Assertion
<checkbox label="q3" atleast="1" optional="0">
    <row label="r1">Apples</row>
    <row label="r2">Bananas</row>
    <row label="r3">Coconuts</row>
    <row label="r4">Grapes</row>
    <row label="r5">Watermelons</row>
    <comment>Select all that apply</comment>
    <title>Which of the following does whom like?</title>
    <col label="c1">You</col>
    <col label="c2">Your spouse</col>
  <checkbox label="vq3" title="vq3" onLoad="copy('q3', rows=True)">
for aRow in data.rows:
    if q3[aRow].c1 or q3[aRow].c2:
        aRow.val = 1
must test vq3

assert vq3r*=1
requires any q3r${1}c1=1 q3r${1}c2=1

Example 3

The below provides code example of using wildcards in assertions.

assert var1_*=val1
requires any var2_${1}=val2 var3_${1}=val3
Requires each var1_X to have one of var2_X=val2 or var3_X=val3

Whatever the wildcard matches (e.g., "_1" will be inserted for the wildcard in the dependent variables on the right side)

assert var*_*=val1
requires any var${1}_${2}=val2 var3_${1}=val3


3:  Running the Validation Script

After you have saved the text file containing the assertions to the project directory, you can run the validation script using this syntax:

autofillvalidator [path/to/project] [path/to/test/file/filename].txt

Each test result will have one of the following results:

Results Description
PASS The conditions returned true for all data records
FAIL The conditions returned false for at least one data record
NO RESULTS There was no data to verify the conditions

Tip: Non hidden/virtual questions can also be tested but will not be included in the test coverage calculation described below.

3.1:  Test Coverage

The test coverage returns the percentage of variables* tested / total variables* where "variables*" meet these conditions:

  • Belongs to a non-transient question (e.g., not record [not a question] or vos [transient])
  • Belongs to a virtual or question where where=survey is not true (i.e., it is not displayed to respondents)

The test coverage percentage does not reflect of coverage of all values for all variables.  When the test coverage does not meet 100% the output will include a list of untested variables.

Tip: A blank rules file can be used to generate a list of all the hidden/virtual questions.

Below is an example of the returned output.

Setting up tests...

Running tests...

AGEGEN=1 requires all q1=1 q2=2 :: PASS
AGEGEN=2 requires all q1=1 q2=3 :: PASS
AGEGEN=3 requires all q1=1 q2=4 :: PASS
AGEGEN=4 requires all q1=1 q2=5 :: PASS
AGEGEN=5 requires all q1=1 q2=6 :: PASS
AGEGEN=6 requires all q1=2 q2=2 :: PASS
AGEGEN=7 requires all q1=2 q2=3 :: PASS
AGEGEN=8 requires all q1=2 q2=4 :: PASS
AGEGEN=9 requires all q1=2 q2=6 :: FAIL
AGEGEN=10 requires all q1=2 q2=6 :: PASS
vq3r1=2 requires any q3r1c1=1 q3r1c2=1 :: NO RESULT
vq3r2=2 requires any q3r2c1=1 q3r2c2=1 :: NO RESULT
vq3r3=2 requires any q3r3c1=1 q3r3c2=1 :: NO RESULT
vq3r4=2 requires any q3r4c1=1 q3r4c2=1 :: NO RESULT
vq3r5=2 requires any q3r5c1=1 q3r5c2=1 :: NO RESULT
vq5r1=1 requires any q5=1 q5ar1=1 :: PASS
vq5r2=1 requires any q5=2 q5ar2=1 :: PASS
vq5r3=1 requires any q5=3 q5ar3=1 :: PASS
vq5r4=1 requires any q5=4 q5ar4=1 :: PASS
vq5r5=1 requires any q5=5 q5ar5=1 :: PASS

Test coverage: 100.0%

Validation FAILED. 1 of 20 tests failed.

4:  Limitations in Assertions

The below provides code examples of the limitations of assert(...).

Example 1

assert var1=val1
requires any var2=val2 var3=val3 …
  • Limitations: This only works one way; this rule by itself would not catch a virtual that codes var1=badval when var2=val2, so you would probably need to create a rule for every possible outcome of the hidden question.
  • Python:
if var1 == val1:
    var2 == val2 or var3 == val3

Example 2

assert var1=val1
requires all var2=val2 var3=val3 …
  • Limitation: Again, this rule would allow the condition 'var2=val2 and var3=val3' to result in var1=badval
  • Python:
if var1 == val1:
    var2 == val2 or var3 == val3