[IDS] review of proposed "terms of reference" for Common Criteria Technical Communities

[IDS] review of proposed "terms of reference" for Common Criteria Technical Communities

[IDS] review of proposed "terms of reference" for Common Criteria Technical Communities

Brian Smithson bsmithson at ricohsv.com
Thu Dec 8 19:36:25 UTC 2011

At the 12th ICCC, I was asked to share some of the hardcopy devices
community's experience and recommendations for technical community terms of
reference. I submitted them to Dag Ströman, the head of the Swedish CC
scheme and also chair of the CC Management Board, and to Matsutoshi Murata
of the Japanese CC scheme. I have not yet received any feedback.

What I submitted was:

 1. An outline of the IEEE P2600 Working Group that developed the Hardcopy
    Devices PPs; and
 2. My recommendations for CCDB Collaborative TC ToRs, based on experience
    with that group, the CCDB Vision paper, discussions with others, David
    Martin's presentations, etc.

For each, I used the same topical outline (except for Issues and Results, in
the case of IEEE P2600).

I had hoped to make a very short set of ToR recommendations, but there
really are quite a few things to address. I've tried to limit the number of
"musts" and used "shoulds" wherever I could. But it's not as bad as it looks.

I am hoping that something along the lines of this proposal will be accepted
by the CC Development Board as the way to organize Technical Communities for
developing Protection Profiles and  Supporting Documents. But who knows?
Please send me your comments...

By the way, we're probably going to start setting up a Technical Community
for hardcopy devices in January, so this is kind of important.

*IEEE P2600 WG*

  * Initiation
      o It was initiated by a group of vendors who held a pre-formation
        meeting and then applied to the IEEE Standards Association (IEEE-SA)
        to form a working group. Subsequent meetings were as an IEEE working
        group. Web site: http://grouper.ieee.org/groups/2600/
  * Incorporation
      o P2600 is a working group of the IEEE-SA and is therefore
        incorporated under the IEEE-SA.
  * Membership
      o Open to anyone (even non-IEEE members), as individuals. Although
        individuals were usually representing a vendor, IEEE-SA working
        group members participate as individuals and issued standards list
        them as such, without associating them with their employers.
      o Vendor representatives were the core participants, although untold
        others lurked on the mailing list. Sometimes the lurkers would offer
        answers or guidance on topics, so it was useful to have them.
      o Schemes and labs and consultants participated in some meetings, and
        on some occasions we specifically invited them to attend.
  * Meetings
      o We held face-to-face meetings every six weeks. There were some
        special occasions for teleconferences, sometimes involving only an
        ad hoc committee that had been formed and announced in a previous
        meeting. We made few exceptions for the face-to-face requirement --
        generally only for invited guests or for members who would need to
        travel from Asia or Europe.
      o Travel budgets were looser then than now. The bulk of our work was
        completed before the economic collapse in 2009.
  * Technical infrastructure
      o We have a web site and several mailing lists, provided by the IEEE-SA.
  * International-friendliness
      o Most members were from the US, but some from Canada, Europe, and Asia.
      o Most of the Asian vendors used their US representatives to
        participate. I think this was more of a language and cultural
        decision than one of international accessibility.
      o Most of our face-to-face meetings were held in the US or Canada, but
        we did have at least one in Europe and Japan. On those occasions
        when we had teleconference calls, we made sure that they were at an
        acceptable time and day for the expected participants.
  * Organization
      o We had an elected chairperson, vice-chairperson, and secretary.
        These positions had a one-year term with no limits. The chairperson
        need to be an IEEE and IEEE-SA member.
      o We also had volunteer editors.
  * Costs and funding
      o There was no cost to organize and operate the WG, because IEEE-SA is
        supported by memberships and sales of standards.
      o We had to negotiate a license purchase for the PPs so they could be
        made available at no charge and so that derivative works (STs) could
        be created (see below, Ownership).
      o We put out an RFP for PP evaluation, chose a lab, and paid for PP
        evaluation and some consulting help.
      o Most but not all vendors contributed, equally, to cover those costs.
        The benefits of contributing included quotes in press releases from
        the IEEE, easier licensing for derivative works, and listing of
        certified conforming products on the P2600 web site.
  * Policies and procedures
      o We operated under procedures conforming to the rules of the IEEE-SA.
        Details: http://grouper.ieee.org/groups/2600/process/OpProcs.doc
      o This was helpful for operating the WG, decision-making, etc.
      o It was cumbersome for initiating projects within the WG (needed to
        submit a project authorization request to the IEEE Standards Board),
        and even more cumbersome for publication (needed to follow IEEE
        standards formats, go through an editing cycle, and form a separate
        balloting group to review and vote on each PP, then submit the PP to
        the Standards Board for approval as an IEEE Standard -- plus get it
        evaluated and certified per the CC).
  * Decision-making
      o Members who attended two of the most recent four meetings could
        request voting rights. If they did not continue to participate,
        their voting rights could be rescinded.
      o Decisions were made based on simple majority, with some quorum
        rules. Since we held face-to-face meetings, there tended to be only
        one voting member from each vendor.
      o We never really had any very contentions issues that tested the
        individual vote (versus one vote per company).
  * Transparency
      o All work including mailing list archives, draft documents, and
        meeting minutes, were open to the public -- until we got close to
        final drafts, at which time the IEEE asked us to password protect
        the drafts to protect their copyright on the issued standards.
  * PP evaluation
      o PPs were formally evaluated by a lab and overseen by a scheme. They
        were not "evaluated on first use".
      o The evaluations were not performed by the same people who helped
        write parts of the PP, which helped ensure that the evaluations were
        done more objectively.
  * Ownership of work product
      o As mentioned before, IEEE-SA held copyright on the PPs and expected
        to sell them. We needed to negotiate a special license to allow
        their free use for CC purposes, but the copyright even with that
        license restricts other uses.
  * Legal protection
      o As a recognized SDO, WG members were protected from being accused of
        collusion. Each meeting was opened with a standard presentation
        slide and statement about topics that were inappropriate to discuss
        in the meetings.
  * Intellectual property
      o We never really worried about revealing trade secrets. The primary
        issue of intellectual property was patents.
      o The WG followed IEEE-SA (which follows ANSI) policies about
        essential patents. Members had a duty to reveal any knowledge of
        patents that were essential to conforming to the standards under
  * Sustainability
      o The issue of work product ownership causes us to consider
        alternatives for future work, but if we wanted to continued under
        IEEE-SA we could do so indefinitely. IEEE has been around for a very
        long time.
  * Issues
      o Work product ownership was the biggest issue. We were unprepared for
        the cost of purchasing the necessary copyright license to make the
        PPs available for use.
      o Conforming to the IEEE Standards approval process was also
        challenging, but at the time we did want to have some recognition of
        the "officialness" of our PPs, so it did serve that purpose.
      o We paid for evaluation lab services (and a small fee to BSI for
        certification). The main issue was that we could not get firm quotes
        from labs until we had nearly completed the PPs, which meant that we
        could not inform the vendors about costs at the start of the
        project. However, we did a little survey of some labs to get a rough
        idea of how much PP evaluation might cost, and that did turn out to
        be pretty close to what we paid.
  * Results
      o We produced a general standard (available for purchase), four
        evaluated PPs, two of those PPs were certified (one by NIAP, the
        other by BSI, both used atsec as the evaluation lab).
      o We also produced an informative guide for writing STs based on those
      o All are available here:
      o As of today, twelve certificates have been published on the CC
        portal for conforming products, an additional six are completed but
        published only on the scheme web site, and nine more are currently
        listed as being in evaluation. Each one of these evaluation projects
        typically represents several MFP product models.

*Recommendations for CCDB-approved ToRs*

  * Initiation
      o Informal "pre-formation" meetings should be used to attract vendors
        and labs and schemes to the possibility of forming a TC. They can be
        called by any stakeholder.
      o Note: I don't know what the process or criteria will be for the CCDB
        to approve the formation of a TC, but I imagine that the criteria
        would include:
          + At least one scheme must be committed to be the sponsor. That
            scheme should have some history of certifying products in the
            TC's technology area.
          + More than one scheme should participate, to help ensure mutual
            recognition of the output.
          + A majority of vendors must be committed to participate. How
            exactly to determine that may be a problem. If some significant
            vendors decline to participate, the CCDB should try to find out
            if they are doing so out of passiveness or due to a specific
          + At least one lab must be committed. That lab should have some
            history in evaluating products in the TC's technology area.
          + More than one lab should participate.
          + The proposed TC must follow the required and recommended ToRs.
      o The formation and approval process itself must be open, with public
        announcements of intent and milestones and such.
  * Incorporation
      o The TC must be incorporated as a legal entity that can hold copyright.
      o The TC should be recognized as a standards development organization
        (e.g., be incorporated under an SDO) to provide anti-trust protection.
  * Membership
      o Membership must be freely open to anyone with a demonstrable
        interest in subject matter of the TC.
          + The TC should reserve the ability to remove members for
            disruptive or inappropriate behavior.
      o Membership should be categorized by stakeholder role (e.g., scheme,
        vendor, lab,...), organization, and representative(s). These are
        used for decision-making processes.
          + Individual membership (not associated with an organization)
            should be allowed.
      o Membership roles should be accessible to members.
  * Meetings
      o Meetings must be announced and conducted according to pre-defined
        rules. Such rules may be defined by the TC itself. For example:
          + Meetings and their agendas are published in advance of the meeting.
          + Meeting summaries or minutes are published for members.
          + Action items are recorded and tracked.
      o Meetings should be held in English language (as the /lingua franca/
        of the CC).
      o Face-to-face meetings should be accessible to those who cannot
        attend in person, by live telephone, recording, or detailed minutes.
  * Technical infrastructure
      o Minimally, the TC must have a managed email list or online forum,
        and file storage for documents, available to all members and
        restricted from access by others.
  * International-friendliness
      o The TC must accommodate an international membership.
          + Note: Depending on the geographic makeup of the group, this may
            require rotation of meeting times to be fair to meeting
            participants (not necessarily the same as members!).
  * Organization
      o The TC must have a chairperson.
          + Note: I am not sure if the CCDB thinks that a scheme
            representative must chair the group, or if they can delegate to
            an elected member, or if it can be fully open to election. If
            the CCDB insists on a scheme rep as chair, they need to make
            that person sufficiently available to be a responsible and
            responsive chairperson.
      o The TC should have a vice-chair as a backup for the chairperson
      o The TC should have a secretary to take minutes, manage action item
        lists, manage document storage, etc.
          + Note: I have found it to be more consistent if one person
            handles these things. The alternative is to rotate the duty or
            ask for volunteers.
      o .Officers must be elected by a defined voting process with a defined
  * Costs and funding
      o The operating costs of the TC must be borne by member organizations
        (not by each representative).
          + The membership fee structure should accommodate smaller
            organizations and individual memberships by providing reduced fees.
          + Free individual membership should be available, but without
            voting or other rights.
      o Membership fees should be reasonable, for example under US$2,500 per
        year for large organizations.
  * Policies and procedures
      o The TC must have written policies and procedures for key operations.
        For example:
          + Membership
          + Voting rights
          + Election of officers
          + Decision-making
          + Making changes to the policies and procedures
  * Decision-making
      o Voting rights must be granted only to schemes, vendors, and labs.
        One vote per organization (not one per representative).
          + Note: Or at least I think so. But this does bring into question
            "what are the relevant stakeholders?". We seem to ignore end
            customers. Although schemes may represent their respective
            government customers, that representation does not extend to
            enterprise and other customers. What about consultants? What
            about academics?
          + Note: This whole thing gets very tricky. Should all be counted
            equally? If so, vendors likely always outnumber all other
            stakeholder roles. Maybe that is OK, because vendors have a
            vested interest in keeping all other stakeholders reasonably
            happy. Schemes may insist on the ability to override group
            votes. Labs may feel structurally disadvantaged. Or should each
            stakeholder group vote among themselves and then decisions are
            made according to the (three?) stakeholder votes?
          + Note: If schemes insist on being able to override a group vote,
            then I suggest that such an action be the result of a CCDB
            majority vote on the issue. That should slow it down and help
            ensure that schemes don't run amok.
  * Transparency
      o All artifacts such as mailing list archives, draft documents,
        meeting minutes, and action items, must be open to all members.
      o The TC should consider making drafts and minutes accessible to the
  * PP evaluation
      o PPs must be formally evaluated by a lab and overseen by a scheme
        before being made available for conforming product evaluations.
      o PPs should be evaluated by a lab and scheme that is selected based
        on competitive bid.
          + Note: Maybe vendors are the only ones who vote on this.
          + Note: I don't mean to imply either that (1) the lowest bid is
            always accepted or (2) that "we'll do it for free" is
            necessarily rejected. The point is that we want labs to be
            committed to perform the work in a timely and responsive manner,
            which (based on what I've heard in some of the TCs) isn't always
            the case with volunteer PP writing/evaluation efforts.
  * Ownership of work product
      o The TC must hold copyright to the PPs, SDs, or any other work products.
      o The copyright must permit free licensing for specific derivative
        works (e.g. STs, evaluation artifacts, etc.)..
  * Legal protection
      o TC members must be given some protection from anti-trust accusations
        or similar legal actions.
      o Each meeting must be preambled by a standard statement about
        inappropriate topics.
  * Intellectual property
      o The TC must have a policy about disclosure of essential patents.
      o Each meeting must be preambled by a standard statement about such
  * Sustainability
      o The TC must be created with the capability and intent to remain in
        operation indefinitely so that questions can be answered,
        interpretations made, and PPs and SDs reaffirmed or revised as needed.
      o The TC should have a plan to turn its copyright works over to a
        suitable entity (the CCDB? is the CCDB a legal entity?) in the event
        of its disbanding.

One final note: It would be ideal if one entity could be created that could
be used as the home for multiple TCs. Perhaps the best way would be to
establish one TC and after demonstrating that it works pretty well, open it
up to other TCs. The benefits of having one entity for multiple TCs include:

  * One membership (and one membership fee) to deal with.
  * Consistent policies and procedures, and infrastructure.
  * Perhaps most importantly, the ability to collaborate on Supporting
    Documents that can be used by multiple TCs.

Brian Smithson
Security Research, Planning
Advanced Customer Technologies
Ricoh Americas Corporation
bsmithson at ricohsv.com

This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.pwg.org/pipermail/ids/attachments/20111208/415fbc43/attachment-0001.html>

More information about the ids mailing list