Skip to main content

FV Decipher Support

All the topics, resources needed for FV Decipher.

 
FocusVision Knowledge Base

Hooks System

  Requires Decipher Cloud

If you need more examples other than the ones listed below, look at what others have done: allgrep hooks.py "def [function]"

1:  Overview

Create a hooks.py file in the directory that you want special behavior added to and be sure to force all surveys to reload. This file is typically located at the client directory level (e.g. /selfserve/53a/hooks.py) or at the project level (e.g. /selfserve/53a/proj1234/hooks.py).

The file must have one or more global functions defined with certain names. These functions will be called at specific times during Decipher operation, typically with a default value (e.g. the default list of variables or the filename for a downloaded file). Your hook can based on information passed in about the relative behavior. Return the default value or modify it as necessary.

2:  Hooks

When making changes to the hooks.py file, be sure to reload your survey to see those changes take affect. Run the following command in your project's directory: here reload .

2.1:  downloads_filename

The downloads_filename function is called to determine the filename of a data download. It applies both to the filenames directly downloaded and names of files within a "bundle" (e.g. a zip file containing a tab delimited data file, datamap, etc...).

Function Description

def downloads_filename(filename, survey, dtype, contents, layout):
Variable Description
filename Default filename value
survey The survey object for which data is being generated
dtype The main data type. "bundle" for the multi-datatype zip, "excel" for Excel/Excel 2007 downloads, "spss" for the SPSS *.sav, and "data" for data files contained within a bundle.
contents For dtype="bundle": a list of data files (e.g. ["data.tab", "datamap.plain"]). For dtype="data": the type of data (e.g. "fw", "csv", "xls", "xlsx", "sav"). Note: "fw" for fixed-width and "plain" for plain data maps correspondingly.
layout If a layout was used, this is the numeric ID. None if no layout used.

This function should return a string.

Example

def downloads_filename(filename, survey, dtype, contents, layout):
    # custom name to use
    name = "NewName"

    # special naming
    if dtype in ("excel", "spss"):
        return "%s_%s" % (dtype, name)

    elif dtype == "data":
        suffix = { 'fw' : 'txt', 'csv' : 'csv', 'xls' : 'xls', 'xlsx' : 'xlsx', 'sav' : 'sav' }
        return "%s_data.%s" % (survey.basename, suffix.get(contents, 'txt'))

    elif dtype == "datamap":
        return "%s-dmap.%s" % (name, contents)

    elif dtype == "readme":
        return "%s_summary.txt" % survey.basename

    # default value
    return filename

2.2:  onLoad

The onLoad function is called to mutate a survey as it's being loaded. This enables you to completely modify the default values for the survey object (e.g. update <survey> attributes, create virtual questions, etc...).

Function Description

def onLoad(ctx):
Variable Description
ctx The survey object's context. (e.g. path, root, errors, attr, compat, el)

Example

def createValid(ctx):
    # Create a "valid" status code variable
    from hermes.elements import Term, Radio, Row
    if 'valid' in ctx:
        return

    terms = ctx[Term.Term]
    if not terms:
        return
    el = ctx.root.createChild(Radio.Radio, label="valid", title="Valid", values="order")
    el.transient = True
    el.attr['virtual'] = 'fetchMarkerM()'
    el._markers = []

    el.createTransient(Row.Row, label='r0' , cdata="Valid")
    el._markers.append("qualified")

    index = 1

    for x in terms:
        # Unfortunate duplication of Term.py
        cond = x.attr.get('cond','').replace(',', ' ')
        if not cond: continue           # no cond= specified for some reason?
        el._markers.append('term:'+cond)
        el.createTransient(Row.Row, label='r%d' % index, cdata=x.cdata)
        index += 1

    # Other terminates
    el.createTransient(Row.Row, label='r%d' % index, cdata='Unspecified terminates', value="98")
    el._unspecifiedIndex = index
    index += 1
    el._markers.append('_NEVER_SET')

    # Overquota
    el.createTransient(Row.Row, label='r%d' % index, cdata='Overquota', value="99")
    el._markers.append('OQ')


    el.notifyEnd(0)

def createVStatus(ctx):
    from hermes.elements import Radio, Row
    if 'vStatus' in ctx:
        return
    el = ctx.root.createChild(Radio.Radio, label="vStatus", title="Status Disposition")
    el.transient = True
    el.attr['virtual'] = '''
if "qualified" in markers:
  data[0][0] = 2
elif "OQ" in markers:
 data[0][0] = 1
else:
 data[0][0] = 0
'''

    el.createTransient(Row.Row, label='r1' , cdata="Term")
    el.createTransient(Row.Row, label='r2' , cdata="OQ")
    el.createTransient(Row.Row, label='r3' , cdata="Quals")
    el.notifyEnd(0)

def onLoad(ctx):
    createValid(ctx)
    createVStatus(ctx)
    ctx.root.attr['spssSimpleCheckbox'] = '1'
    ctx.root.attr['fwoe'] = 'text,textarea,other'

2.3:  live

The live function is called when a survey is launching.

Function Description

def live(path, **kw):
Variable Description
path The survey's relative path (e.g. selfserve/9d3/proj1234)
**kw Usually an empty dictionary

Example

import os

# remove variables.xls and cowsay "You are LIVE"
def live(path, **kw):
  try:
    os.remove("%s/variables.xls" % path)
  except OSError:
    pass
    print """
 ______________
< You are LIVE >
 --------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||
    """

2.4:  closed

The closed function is called when a survey has been closed.

Function Description

def closed(path):
Variable Description
path The survey's relative path (e.g. selfserve/9d3/proj1234)

Example

from hermes import misc

# tighten up OE field widths when survey has closed
def closed(path):
    misc.runScript("tighten-widths %s oe" % path)

2.5:  fields

The fields function is called to determine the list of variables for a survey.

Function Description

def fields(vl, survey):
Variable Description
vl The default list of variables which can be reordered, disabled, etc...
survey The survey object for which data is being generated

Example

import os

def fields(vl, survey):
    # return default list if variables exists
    if os.path.isfile('%s/variables.xls' % survey.path):
        return vl

    # hide ipAddress from data
    for x in vl:
       if x.label == 'ipAddress':
            x.notdp = True
    return vl

2.6:  variables

The variables function is called when loading variables from the variable.xls file, enabling you to modify or delete them.

Function Description

def variables(vl, survey, **kw):
Variable Description
vl The default list of variables which can be modified or deleted
survey The survey object for which data is being generated
**kw Usually an empty dictionary

Example

def variables(vl, survey, **kw):
    for v in vl:
        if v.s.type == "text" and v.type == "data":
            v.s.fwidth = 256
    return vl

This function sets the variable group's field width (v.s.fwidth) to 256 for all question types (v.s.type) that are "text" and the data category (v.type) is "data". Other categories include: oe, uuid, date, markers, record, status or extra.

2.7:  sav_variables

The sav_variables function is called to transform a variable list into a list of variables for SPSS *.sav generation. This essentially works on a dataset similar to the output of "generate survey all savdict", so it can be used to adjust SPSS variable names and values.

Function Description

def sav_variables(vl, survey):
Variable Description
vl The default list of variables which can be reordered, disabled, etc...
survey The survey object for which data is being generated

Example

def sav_variables(vl, survey):
    for x in vl:
        var = survey.root.variableDict[x['v2label']]
        if var.s.q and var.s.q.type.rating:
            for i, v in enumerate(x['values']):
                m = NumRx.search(v[1])
                if m:
                    val = int(m.group(1))
                    x['values'][i] = (v[0], '%d %s' % (val, NumRx.sub('', v[1])))
    return vl

2.8:  spss_variables

The spss_variables function is called immediately after the system processes the variables to generate the *.spss script file.

Function Description

def spss_variables(vl, **kw):
Variable Description
vl The default list of variables which can be reordered, disabled, etc...
**kw Usually an empty dictionary

Example

def spss_variables(vl, **kw):
    for v in vl:
        var, rest = v['title'].split(':', 1)
        if ' - ' in rest:
            rest = rest.split(' - ',1)[0]
        v['title'] = '%s.%s' % (var, rest)
    return vl

The function above modifies titles like "variable: This is the variable" to "variable. This is the variable". In addition to the colon -> dot update, if the title has multiple parts (e.g. "Green - which colour do you like"), only the first part will be kept (e.g. "Green").

2.9:  spss_base_title

The spss_base_title function generates a title for a variable without label prefixed.

Function Description

def spss_base_title(v):
Variable Description
v The variable to modify

Example

def spss_base_title(v):
    "Title without label prefixed"
    return v.title or v.s.title

2.10:  spss_title

The spss_title function generates a title for a variable with a label prefixed.

Function Description

def spss_title(v, label, title):
Variable Description
v The variable to modify
label The element's label
title The element's title

Example

def spss_title(v, label, title):
    "title with label"
    return '%s. %s' % (label, title)

2.11:  checkin

The checkin function runs when a file has been checked. It can be used to notify users of changes in files tracked by the version control system (e.g. survey.xml, nstyles, styles, language files, etc...).

Function Description

def checkin(filename, added, deleted, username, old):
Variable Description
filename The name of the file being altered
added The count of added names
deleted The count of deleted names
username The name of the user who changed the file (not always accurate)
old The content of the old file version

Example

import os.path, time
from hermes import misc, gv

"""
Checkin hook for sending email on live survey changes.

Add this to a hooks.py file in the top level directory; all subdirs
will be affected.

When a live survey has its survey.xml or language.xml or nstyles file
modified, an email will be sent to to users listed in the notify.txt file.

The file must contain exactly one email address in the form user@domain.com per line.

If the file does not exist, notifications are disabled for that directory.

Surveys with "temp" in any part of their name do not have notifications enabled.
"""


def checkin(filename, added, deleted, username, old):
    # eligible file?
    dirname, basename = os.path.split(filename)
    if not basename.endswith(".xml") and basename != "nstyles":
        return

    # Email list specified?
    try:
        emails = filter(None, map(str.strip, open("%s/notify.txt" % dirname)))
    except IOError:
        return

    # survey live?
    if misc.getSurveyAttribute(dirname, "state") != "live":
        return

    # copy-survey to temp directory will copy the notify.txt too
    if 'temp' in dirname:
        return

    # survey was previously in testing, and now went live
    if basename == "survey.xml" and old and ('state="testing"' in old or "state='testing'" in old):
        return

    title = misc.getSurveyAttribute(dirname, "alt") or misc.getSurveyAttribute(dirname, "title")

    emails = ', '.join(emails)
    if username == 'user':
        username = '(unknown)'

    mtime = time.ctime(os.path.getmtime(filename))
    secure = gv.secureHost

    misc.sendRawMail("""From: <nobody@surveys.client.com>
To: %(emails)s
Subject: [%(dirname)s] Survey modified

The file %(basename)s in directory %(dirname)s has been modified.

Likely by: %(username)s

Lines added: %(added)s
Lines removed: %(deleted)s
File changed on: %(mtime)s

Report link: %(secure)s/report/%(dirname)s
Review exact changes at: %(secure)s/admin/vc/list?file=%(filename)s

Note: username may not be exact depending on how file was modified.
Actual file was changed on the time above, but notification happens only when
accessed by the survey system.

--

    """ % locals())

2.12:  layout_variables

The layout_variables function runs when a layout has been selected and the variable list is ready to be used. This can be used to customize the data downloaded.

Function Description

def layout_variables(vl, survey, layout, path):
Variable Description
vl The default list of variables
survey The survey object for which data is being generated
layout The ID of the layout being generated
path The survey's relative path (e.g. selfserve/9d3/proj1234)

Example

# only Uncle data downloaded has any of the _shown variables
def layout_variables(vl, survey, layout, path):
    if 'what=uncle' not in path:
        return [x for x in vl if x.s.label != '_shown']
      else:
        return vl

2.13:  wrangle_select

The wrangle_select function runs every time a tab-delimited or fixed-width data row is being created. This lets you replace or blank out values any way you want, even replacing numbers with special characters. The changes do not affect reporting, Crosstabs, data editor, etc... but they do affect generate script ran from the command line.

Function Description

def wrangle_select(survey, variableList):
Variable Description
survey The survey object for which data is being generated
variableList The list of variables included in the export. The order, labels and contents may vary from survey to survey or from layout to layout.

Example

# This hook is called to determine whether data wrangling occurs on the survey
# NB: there's a performance penalty when enabled

# if you make changes to the hooks.py file, make sure to "here reload ." to force the survey to
# refresh

def wrangle_select(survey, variableList):
    # for the sake of performance, generate a dictionary mapping the variable objects we want to look at
    # to values we'll replace with a blank
    replace = {}

    # What variables are involved in this export? The order, labels and contents
    # may vary from survey to survey or from layout to layout
    for v in variableList:
        # If the variable has a question (i.e. not uuid), and the question is <radio> and there
        #  are exactly 6 item values... mark the last listed item value as one to be
        #  be replaced
        if v.s.q and v.s.q.xmlTagName == 'radio' and len(v.s.itemValues) == 6:
            # the itemValues is a list of how we convert each database value (starting at 0) in
            # downloads
            replace[v] = str(v.s.itemValues[-1].storedValue)

    # This function will be called before generating each tab delimitered row
    def replaceTab(d):
        for v, value in replace.items():
            if d[v] == value:
                # modify this variable's value to something silly if we get a match
                d[v] = "XXX"

    # this function is called when generated a fixed-width row
    def replaceFw(d):
        for v, value in replace.items():
            # Note that the variable may be space-padded in a fw- file depending
            # on fixed-width assigned to the variable
            if d[v].strip() == value:
                # Generate one X for each fixed-width data file character this variable is assigned
                # NOTE: if you create less than v.fw.width data, the result is padded
                # NOTE: if you create more, a fatal error occurs
                d[v] = 'X' * v.fw.width


    # this function must return two items, either of which can be None
    # which are the functions to call back for tab and FW data generation
    return replaceTab, replaceFw

2.14:  Triple-S

The following two functions can be used together to customize the output of the Triple-S generated data.

2.14.1:  sss_final

The sss_final function runs when the survey XML has been generated in memory and allows you to either modify the XML or return a string. If a string is returned, then the string is written as given. 

Function Description

def sss_final(doc, survey, layout):
Variable Description
doc This is the survey XML object
survey The survey object for which data is being generated
layout If a layout was used, this is the numeric ID. None if no layout used.

Example

"""
Version History:

v2: Dec 15:
 - Stripped first 2 path segments off <name>
 - decipher:title uses report/alt title
 - decipher:subtitle added for 2D questions
 - decipher:loopid shows loop's label for expanded questions
 - decipher:looprow shows the looprow's label
 - decipher:lopoindex shows 1-based looprow offset
 - decipher:shape is set to "grid" if question has both multiple cols and rows
"""

# This is called when the XML has been generated in memory
# doc: this is the XML object
# survey: the survey
# layout: ID of layout used if any
def sss_final(doc, survey, layout):

    # Change the version attribute on the document to 2.0
    doc.attr.version = '2.0'

    # Add an XML namespace so that attributes that are named decipher:something are legal
    setattr(doc.attr, 'xmlns:decipher', 'http://decipherinc.com/sss')

    # strip the initial 2 path components
    doc[4][0][0] = '/'.join(survey.path.split('/')[2:])

2.14.2:  sss_var

The sss_var function runs after every variable XML has been generated. This is the <variable> tag in the XML.

Function Description

def sss_var(survey, var, el, q):
Variable Description
survey The survey object for which data is being generated
var The <variable> element
el The survey element (e.g. radio, checkbox, etc...)
q The question element

Example

"""
Version History:

v2: Dec 15:
 - Stripped first 2 path segments off <name>
 - decipher:title uses report/alt title
 - decipher:subtitle added for 2D questions
 - decipher:loopid shows loop's label for expanded questions
 - decipher:looprow shows the looprow's label
 - decipher:lopoindex shows 1-based looprow offset
 - decipher:shape is set to "grid" if question has both multiple cols and rows
"""


# This is called after every variable XML has been generated
# this is the <variable> tag in the XML

def sss_var(survey, var, el, q):
    ":type q: QuestionElement"

    # If we generated this variable for a question...
    if q is not None:

        # Set formname to variable label if two-dimensional question
        if len(q.rows) > 1 and len(q.cols) > 1:
            setattr(el.attr, 'decipher:formname', var.reportLabel.split("_")[0])

        # Set formname to question label if one-dimensional question
        else:
            setattr(el.attr, 'decipher:formname', q.label.split("_")[0])

        # Set formlabel to variable text if two-dimensional question
        if var.label not in ('interview_start', 'interview_start_time', 'interview_end', 'interview_end_time'):
            if len(q.rows) > 1 and len(q.cols) > 1:
                if q.grouping.rows:
                    setattr(el.attr, 'decipher:formlabel', q.title.decode('utf8'))
                else:
                    setattr(el.attr, 'decipher:formlabel', var.title.decode('utf8'))

            # Set formlabel to question label if two-dimensional question
            else:
                setattr(el.attr, 'decipher:formlabel', q.title.decode('utf8'))

        # Set formtype to question type
        if (len(q.rows) > 1 and len(q.cols) > 1 and q.grouping.rows) or q.label == "Q35NEW":
            setattr(el.attr, 'decipher:formtype', "grid")
            try:
                setattr(el.attr, 'decipher:precode',  var.reportLabel.split("_")[1])
            except IndexError:
                setattr(el.attr, 'decipher:precode',  var.reportLabel.split("_")[0])
        else:
            if q.xmlTagName == "radio":
                setattr(el.attr, 'decipher:formtype',  "single")
            elif q.xmlTagName == "checkbox":
                setattr(el.attr, 'decipher:formtype',  "multi")
            elif q.xmlTagName == "number":
                setattr(el.attr, 'decipher:formtype',  "numericlist")
                try:
                    setattr(el.attr, 'decipher:precode',  var.reportLabel.split("_")[1])
                except IndexError:
                    setattr(el.attr, 'decipher:precode',  var.reportLabel.split("_")[0])
            else:
                setattr(el.attr, 'decipher:formtype',  q.xmlTagName)

        # Set fixtype to Position - ConfirmIt Spec
        setattr(el.attr, 'decipher:fixtype',  'Position')

        # Set source to survey path
        setattr(el.attr, 'decipher:source', survey.path)

        # Set parent attributes for two-dimensional questions, grouped by columns
        if len(q.rows) > 1 and len(q.cols) > 1 and q.grouping.cols:
            # Set parenttype to 3Dgrid for loop questions that are two dimensional
            setattr(el.attr, 'decipher:parenttype', '3Dgrid')

            # Set parentname to original question label
            setattr(el.attr, 'decipher:parentname', q.label.split("_")[0])

            # Set parentlabel to question title
            setattr(el.attr, 'decipher:parentlabel', q.reportTitle.decode('utf8'))

        # if question was created from a loop template...
        if q.templateLoopvar:
            # Only set loop attributes if real loop questions, NOT virtuals
            if not (q.templateLoopvar.parent.label.decode('utf8').startswith("VIRTUAL")):
                # Set loopid to loop label
                setattr(el.attr, 'decipher:loopid',  q.templateLoopvar.parent.label.decode('utf8'))

                # Set loopiterationid to looprow label
                setattr(el.attr, 'decipher:loopiterationid', q.templateLoopvar.label.decode('utf8'))

            if len(q.rows) > 1 and len(q.cols) > 1:
                # Set formname to report label
                setattr(el.attr, 'decipher:formname', var.reportLabel.split("_")[0])

                if q.grouping.rows:
                    setattr(el.attr, 'decipher:formtype', "grid")

                # Set formlabel to variable title
                setattr(el.attr, 'decipher:formlabel', var.title.decode('utf8'))

                # Set parent attributes for two-dimensional questions, grouped by columns
                if len(q.rows) > 1 and len(q.cols) > 1 and q.grouping.cols:
                    # Set parenttype to 3Dgrid for loop questions that are two dimensional
                    setattr(el.attr, 'decipher:parenttype', '3Dgrid')

                    # Set parentname to original question label
                    setattr(el.attr, 'decipher:parentname', q.label.split("_")[0])

                    # Set parentlabel to question title
                    setattr(el.attr, 'decipher:parentlabel', q.reportTitle.decode('utf8'))
            else:
                # Set formname to report label
                if "r" in var.reportLabel:
                    setattr(el.attr, 'decipher:formname', var.reportLabel.split("r")[0])
                else:
                    setattr(el.attr, 'decipher:formname', q.label.split("_")[0])

                # Set formlabel to variable title
                setattr(el.attr, 'decipher:formlabel', q.title.decode('utf8'))

                if q.templateLoopvar.parent.label.decode('utf8').startswith("VIRTUAL"):
                    setattr(el.attr, 'decipher:parenttype', '3Dgrid')

                    # Set parentname to original question label
                    if "x" in q.label:
                        setattr(el.attr, 'decipher:parentname', q.label.split("x")[0])
                    else:
                        setattr(el.attr, 'decipher:parentname', q.label.split("_")[0])

                    # Set parentlabel to question title
                    setattr(el.attr, 'decipher:parentlabel', q.reportTitle.decode('utf8'))


    # If the variable was date/start_date, change the type to "date"
    # (the default of text is "character")
    if var.label in ('interview_start', 'interview_end'):
        el.attr.type = 'date'
        setattr(el.attr, 'decipher:formtype', el.attr.type)
        setattr(el.attr, 'decipher:formname', var.label)
    elif var.label in ('interview_start_time', 'interview_end_time'):
        el.attr.type = 'time'
        setattr(el.attr, 'decipher:formtype', el.attr.type)
        setattr(el.attr, 'decipher:formname', "_".join(var.reportLabel.split("_")[0:-1]))

    # Add some other attribute with the survey path
    setattr(el.attr, 'decipher:source', survey.path)

2.15:  filtered_report

The filtered_report function allows you to force a custom filter every time a specific user runs a report. This allows you to restrict what subset of the project's data is visible to users.

Function Description

def filtered_report(user, survey):
Variable Description
user The user object representing the user accessing the data
survey The survey object for which data is being generated

Example

from hermes import misc

def filtered_report(user, survey):
    if user.isShared:
        if user.name.startswith("store_"):
            return "store.r%s" % user.name.split("_")[1]
        else:
            return misc.denyAccess("this report -- bad username")

2.16:  subscribers

The subscribers function allows you modify the list of email addresses that receive notifications for your project's events.

Function Description

def subscribers(l, intention):
Variable Description
l The list of email addresses currently receiving notifications
intention The type of notification and reason for sending out the notification
  • Support Requests
    • intention == "support"
  • Fatal Errors
    • intention == "error"
  • Team Emails
    • intention == "team"
    • Sent when setting the survey LIVE, closing the survey, or editing the data

Example

def subscribers(l, intention):
    if intention == "support":
        l.append("client-support@client.com")
    return l

In the example above, "client-support@client.com" will receive a notification for all support requests.

Test your hook to confirm it's working properly. If configured improperly, emails will be sent to the de-support@focusvision.com address instead.

2.17:  campaign_uploaded

The campaign_uploaded function allows you notify users when a list is uploaded via the Campaign Manager.

Function Description

def campaign_uploaded(path, user, data):
Variable Description
path The survey's relative path (e.g. selfserve/9d3/proj1234)
user The user object representing the user accessing the data
data A dictionary containing various information on the project

Example

def campaign_uploaded(path, user, data):
        misc.sendRawMail("""From: <nobody@decipherinc.com>
To: client-support@client.com
Subject: [%s] list uploaded

List %s has been uploaded

Number of records: %d

User uploading: %s

File uploaded on: %s

Name of campaign: %s

--

    """ % ( path, data['fileName'], data['counts']['total'], user.email, time.strftime("%A,%B, %d, %Y at %I:%M:%S"), data['campaign']['name']))

In the example above, "client-support@client.com" will receive a notification when a list is uploaded through the Campaign Manager. The notification will contain various information on the list.

2.18:  verify_survey

The verify_survey function lets you add additional restrictions to contents of the survey (i.e., forbid or require some attributes).

Function Description

def verify_survey(survey, errors):
Variable Description
survey The survey object for which data is being generated
errors Object to display errors

Example

def verify_survey(survey, errors):
  if self.root.compat >= 122 and not self.root.client:
    errors.append((self.root, “The client attribute is required”))

In the example above, the survey will error if the compat level is 122 or above and the client attribute is not present.

2.19:  survey_environment

The survey_environment function runs when the survey is being loaded and enables you to change the functions defined within the survey. This hook works similar to an <exec when="init">.

Function Description

def survey_environment(env, survey):
Variable Description
env The survey environment in which we are adding Python functions to
survey The survey object for which data is being generated

Example

def my_function(x):
    return x * 2

def survey_environment(env, survey):
    env["my_function"] = my_function

In the example above, the method my_function is added to the survey environment and can be called within any survey affected by the hook. For example:

<number label="vQ2" title="Doubled value of Q2" size="3">
    <exec>vQ2.val = my_function(Q2.ival)</exec>
</number>

It is recommended that you use re-loadable modules to explicitly and carefully expose what functions are imported into the secure environment. Please never use execfile.

To use re-loadable modules, you can, for example, do the following:

  • Create local/__init__.py to make the local directory a Python package
  • Create local/mylib.py to add a Python module
  • Create a hooks.py file containing the following:
def survey_environment(env, survey):
    from local import mylib
    reload(mylib)
    env.update(mylib.exported)

And add the following to mylib.py:

import os
from hermes import gv

def staticFiles():
    return os.listdir("%s/static" % gv.survey.path)

def myfunc():
    return gv.survey.path.upper()

exported = dict(myfunc=myfunc, staticFiles=staticFiles)

In the example above, the survey environment hook will ensure that the local.mylib module is reloaded (without it, it would only get imported once). Then, every function (and only those functions) in the exported dictionary are added to the current survey's environment. The exposed functions list the static files for the currently executing survey and return the current survey path in upper-case as another example. This illustrates how you can create methods that wouldn't normally be supported in a secure environment (e.g. os.listdir is not normally allowed).

The functions declared in hooks.py do not have access to functions like setMarker. You can, however, use gv.survey.env["Q3"] to access Q3 or another name just as you would within the survey XML.

Any survey_environment hooks on a multi-tenant server must undergo a security audit by FocusVision Development Operations Staff. For single-tenant servers, a security review may be requested for any hook that you wish to add.

2.20:  request_allowed

The request_allowed function runs when using the v2SendRequest Python function in a secure survey. It is used to validate which remote domains are allowed to send requests to.

Function Description

ef request_allowed(path, domain):
Variable Description
path The survey's path (e.g. selfserve/9d3/proj1234)
domain The domain name of the server where the request is being submitted to

Example

def request_allowed(path, domain):
    return domain in {"api.client.com", "somedomain.net"}

In the example above, the v2SendRequest method will only submit the request if the domain of the URL is found in the set literal declared, containing "api.client.com" or "somedomain.com". To use v2SendRequests, you must specify the exact list of domains to use.

Learn more: v2SendRequests

2.21 portal_restrictions

The portal_restrictions function is called to restrict a user's portal access. 

The below code MUST be inserted by Decipher staff.

As a reminder, this function requires  Decipher Cloud access and should be placed in data/hooks.py (not a client folder).

Function Description

def portal_restrictions(user):
Variable Description
user The user account to restrict portal access.

This hook returns a dictionary with the following keys.

Key Value Description
canViewUsersInSearch True/False When set to false, the portal hides the users/user group column in the advanced search, and does not autocomplete users in the Search box.
canViewTagsInSearch True/False When set to false, the portal hides tags column in the advanced search, and does not autocomplete tags in the Search box.
canViewUsersTab True/False When set to false, the portal hides the users tab in project detail view.
canCreateCompanies True/False For staff or supervisor: When set to false, disables user from creating companies in the portal. Hides the Create New Company button.
canEditCompanySettings True/False For staff or supervisor: When set to false, disables user from making edits to companies in the portal. Hides the Edit Company button.

Example

The following example demonstrates placing restrictions so that only users with emails ending in @decipherinc.com can create and edit companies.

from fnmatch import fnmatch

CREATE_COMPANIES_MATCHER = "*@decipherinc.com"

EDIT_COMPANY_SETTINGS_MATCHER = "*@decipherinc.com"

def portal_restrictions(user):

    ret = {}

    if not fnmatch(user.email, CREATE_COMPANIES_MATCHER):

        ret['canCreateCompanies'] = False


    if not fnmatch(user.email, EDIT_COMPANY_SETTINGS_MATCHER):

        ret['canEditCompanySettings'] = False

    return ret

2.22:  project_created

The project_created function is called when a project is created or duplicated from the portal.

Function Description

def project_created(user, path, source, xml):
Variable Description
user The user object that is doing the copying.
path The path to the new survey. 
source Set to None for a new project.  Otherwise, the path to the original project.
xml An object used to modify content of the survey when a project is duplicated.  The syntax for usage:
with xml() as root:
  root.set("[attribute]", "[value]")

About Setting Preferences

Survey Editor preferences are set per user and if a user creates projects in multiple directories (i.e. company with subfolders), where both directories contain a project_created hook with differing preferences, note that both preferences are set. 

In addition, not all of the Survey Editor's preference keys match the survey.xml. Using an incorrect reference to a preference  in hooks.py causes none of the preferences to be applied and the user's whiteboard, where preferences are stored, may also need to be cleared in the database.  

Therefore, to see what key/value pair to use:

First, open the project settings and select the desired preferences.  Next, run "Fabric._whiteboard.get()" in the browser console to see the key/value pair to use.

Project Settings are only saved and applied after clicking the "Build" button when creating a project. Clicking "Save & Return to Project Page" will not trigger preferences.

Path when Using a Theme

The following applies to the theme path when setting "lumos.style.theme".

  • For a system theme, set the path to: theme-static/nthemes/system/[themefolder]
  • For a client specific theme, set the path to: theme-selfserve/[clientdir]/[themefolder]

For a client specific theme, only users that are part of the company have access to the company theme and if attempting to set "lumos.style.theme" to a client specific theme for a user that is not part of the company, an error results in the Survey Editor (i.e., your staff account). 

Example

The following example demonstrates setting all Survey Editor preferences when a project is created and when a project is copied, enabling "Auto Recover Data", "Global Fir", a logo and custom client theme.

from hermes.misc import ensureDirectoryExists
import shutil, os
SS = '{http://decipherinc.com/ss}'
 
def project_created(path, user, source, xml):
    ":type user: hermes.User.User"
    prefs = user.getPref("builder")
   
    # Display Settings
    ## ss:logoFile
    prefs["lumos.logo.src"] = "selfserve/[clientdir]/[logoFile]"
    ## ss:logoPosition
    prefs["lumos.logo.position"] = "left"
    prefs["lumos.fir.firGlobal"] = "on"
    prefs["lumos.fir.firStyle"] = "square"
    prefs["lumos.fir.firSize"] = "25px"
    prefs["lumos.fir.firColors"] = "#756f75,#c7b1c7,#dfe3b6,#1c6664"
    prefs["lumos.style.theme"] = "theme-selfserve/[clientdir]/[themefolder]"
    ## support links - xml style
    prefs["styleOverrides"] = [{"TEXT": "<div style=\"text-align: center;\"><strong>&nbsp;<a href=\"/support\" target=\"_blank\">Help</a></strong></div><p></p><p></p>", "attr": {"type": "style", "cond": "1", "id": "catZ7","name": "survey.respview.footer.support"}}]
    prefs["lumos.listDisplay"] = "0"
   
    # Device Settings
    prefs["lumos.mobile.mobileDevices"] = ["smartphone", "tablet"]
    prefs["lumos.mobile.mobile"] = "mobileOnly"
    prefs["lumos.mobile.desktopNotAllowedMessage"] = "CUSTOM The device you are using is not allowed to take this survey."
    prefs["lumos.mobile.featurephoneNotAllowedMessage"] = "CUSTOM The device you are using is not allowed to take this survey."
    prefs["lumos.mobile.smartphoneNotAllowedMessage"] = "custom message"
    prefs["lumos.mobile.tabletNotAllowedMessage"] = "custom message"
    prefs["lumos.mobile.mobileNotAllowedMessage"] = "custom message"
    prefs["lumos.mobile.surveyDisplay"] = "mobile"
   
    # Question Settings
    prefs["lumos.mandatory.radio"] = 0
    prefs["lumos.instructions.radio"] = "CUSTOM Select one"
    prefs["lumos.mandatory.checkbox"] = 0
    prefs["lumos.mandatory.select"] = 0
    prefs["lumos.instructions.select"] = "CUSTOM Select one for each"
    prefs["lumos.instructions.checkbox"] = "CUSTOM Select all that apply"
    prefs["lumos.mandatory.text"] = 0
    prefs["lumos.instructions.text"] = "CUSTOM Be specific"
    prefs["lumos.mandatory.number"] = 0
    prefs["lumos.instructions.number"] = "CUSTOM Enter a number"
   
    # Field Settings
    ## Survey Browser Title - name
    prefs["lumos.fieldOptions.surveyTitle"] = "My Survey"
    ## Add a "Back" button to the bottom of each page
    prefs["lumos.fieldOptions.showBackButton"] = "1"
    ## Enable Respondents to Navigate Backward Through Survey
    prefs["lumos.fieldOptions.enableNavigation"] = "1"
    ## Enable Browser Back Button
    prefs["lumos.fieldOptions.disableBackButton"] = "1"
    prefs["lumos.fieldOptions.hideProgressBar"] = 1
    prefs["lumos.fieldOptions.showQuestionNumbers"] = "1"
    ##  Remember Respondent's Place - autosave
    prefs["lumos.fieldOptions.autoSave"] =  1
    prefs["lumos.fieldOptions.autosaveKey"] = "source,cookie"
    ## Verify Unique Respondents
    prefs["lumos.fieldOptions.browserDupes"] = "strict"
    ## Force Secure Links
    prefs["lumos.fieldOptions.secure"] = 1
    prefs["lumos.fieldOptions.loggedInCanSubmit"] = 1
    prefs["lumos.fieldOptions.autoRecover"] = 1
    ## Host Survey From A Different Domain - builder:cname
    prefs["lumos.fieldOptions.cnameVal"] = "v2-odd.decipherinc.com"
    prefs["lumos.style.colorScheme"] = "null"
    user.setPref(namespace="builder", preferences=prefs)
 
    # The below will only apply to duplicated surveys.
    if source is not None:
      with xml() as root:
        root.set("autoRecover", "1")
        root.set(SS + "logoFile", "selfserve/[clientdir]/[logoFile]")
        root.set("fir", "on")
    
        try:
          os.remove("%s/static/theme.css" % path)
          os.remove("%s/static/mobile.css" % path)
          os.remove("%s/static/theme.less" % path)
        except OSError:
          pass

        if not os.path.isdir("%s/static" % path):
          os.mkdir("%s/static" % path)

        shutil.copyfile("[path/to/new/theme]","%s/static/theme.less" % path)

2.23:  quota

You can use a quota hook to modify the possible cell assignments for a respondent. 

To make such a custom modification to a quota, you define a quota_hook function inside a <exec when="init">< /exec> code block in the survey.

Within the quota_hook function, you receive the following information:

  •  quota sheet name (e.g. "Sheet1")
  •  table name
  •  list of cells

The list of cells contain the following:

  • fill fraction 
  • set of markers that make up the cell (e.g. male, 18-25 if you had a age*gender quota)
  • the marker that will be set if successful

The quota function can take the cell list and remove, reorder or even add new items. The quota function must return the modified cell list for the quota process to continue. When the quota process continues with the new cell list, for each table: the first marker in the cell list will be set (or the first N markers if using cells:N).

Example

For example, for a survey where a respondent views multiple concepts, you might want to use a quota hook to specify  that the respondent never sees the concept combination of concept1 and concept2.

For the quota hook example, a survey has a defines and two quota sheets as displayed below.

Defines

  A plus
1 one plus
2 two plus
3 three plus
4 never_picked  plus
5 concept1 plus
6 concept2 plus
7 concept3 plus
8 concept4 plus
9 concept5 plus
10 concept6 plus

Sheet1

  A B
1 # = ConceptsA  
2 one 25
3 two 25
4 three 25
5 never_pick 500

Sheet2

  A B
1 # = ConceptsB  
2 concept1 100
3 concept2 100
4 concept3 100
5 concept4 100
6 concept5 100
7 concept6 100

For sheet 1, we never want the never-pick marker to be set.  This is specified in the last line of code in the quota hook below.  We create a new list of cells where x[2] (marker name) is not /never_picked.

For sheet 2, we never want concept1 and concept2 assigned together. To do this, on quota sheet "Sheet2" we go over the eligible list of cells. We find the first of either concept1 or concept2. Then we mark the other quota cell for removal and return all but that quota cell. As a result, only concept1 OR concept2 can ever be picked.

<exec when="init">
C1 = '/Sheet2/concept1'
C2 = '/Sheet2/concept2'
def quota_hook(sheet, table, cells):
  if sheet == "Sheet2":
    remove = None
    for x in cells:
      if x[2] in (C1, C2):
          remove = C1 if x[2] == C2  else C2
          break
    return [x for x in cells if x[2] != remove]
  else:
    return [x for x in cells if x[2] != '/Sheet1/never_picked']
</exec>

2.24:  pii_levels

Through the hooks system, you can define a pii_levels hook that will run when variables are being prepared. This hook can be used to automatically apply PII level to specific questions or variables. For example:

def pii_levels(survey, d):
 for hide in ("ipAddress", "XID"):
   try:
     d[hide].pii = 100
   except KeyError:
     pass

The above hook code would, if ipAddress or XID exist as a name, set their PII level to 100. If variables do not exist, they are ignored. During the PII level setting, any aspect of the question (e.g. specific label names in the XML) can be examined before a decision is made to force a PII level.

To read more about PII click here.

2.25:  user_pii_level 

Any PII level within the 0...9999 range can be applied to any question or variable. This level is then compared to an inherent PII level each user account has, which we suggest will be as follows:

  • 1 -- a shared user account (suggesting that auditing access to who really got the data is not possible)
  • 2 -- a non-shared user account (i.e. created with a specific email)
  • 4 -- supervisor users (Research Hub only)
  • 8 -- staff users

The static level can be modified by a hook or possibly other future configuration. The hook can be implemented as follows:

def user_pii_level(level, user, survey):
  if survey.path.startswith("abc/") and not user.isStaff:
      return min(level, 2)
  return level

The above example hook will consider the survey's path: if the path starts with "abc/" the user's PII level is forced to be at most 2 suppressing normal higher classification.

Other considerations here could check the user's effective level over the survey, for example if the user could edit the data in the survey, his level

The action being performed cannot be used to determine effective PII level.

To read more about PII click here.

2.26:  fileupload_allowed

You can use the fileupload_allowed hook to limit a user's access to upload or delete files in the image manager (survey editor / campaign manager), logo manager, file manager, and video API.

Function Description

def fileupload_allowed(path, user, filename)
Variable Description
path The survey's relative path (e.g. selfserve/9d3/proj1234).
user The user object representing the user accessing the data.
filename Filename currently being uploaded/deleted.

Example

def fileupload_allowed(data, path, user, filename):
  if user.isStaff or path.split('/')[1] == user.selfserveDirectory() and user.canCreateProjects(user.company_id):
    return True # you can upload files
  # anyone can upload .pdf files
  if filename.endswith(".pdf"):
    return True
  return False # anything else not allowed

In the above example, the following permissions are set:

  • Staff users can add and delete files in any project.
  • Supervisors/project creators can add and delete files in projects that are located in the company to which they belong.
  • Regular users with survey:build or edit access are restricted from adding and deleting files.