University of Pittsburgh |  Pitt Home | Find People | Contact Us
 
 
Home
About
Articles & Chapters
Conference Papers
Presentations
CMU Testbed
GWU 2004 Talks
Web Survey Results
Reports & Transcripts
Grant Proposals
Podcasts
Qualitative Data
Analysis Program

eRulemaking Blog
Contact

Revised
01/11/2007
 

 

Answers to Open Ended Questions
{Please note: to print these pages you must first set your browser Page Set Up to "Landscape"}

Question 18: What would you most prefer to change about the electronic commenting/docket system that you used?


It is too difficult to move around in the docket.  It would be most helpful if docket entries were dated and if the names of commenters were displayed.


Integration with the semiannual regulatory agenda, thus improving upon the site at http://ciir.cs.umass.edu/ua/  Integration with any FOIA e-reading rooms with materials related to the rule, like the  single source  pages set up at Labor/MSHA in which a large number of frequently requested docs on a signle subject are grouped together for one-stop shopping  Links to reg impact analyses
Integration with the semiannual regulatory agenda, thus improving on the site at http://ciir.cs.umass.edu/ua/, which is an improvement over the Fed Reg version  Integration with FOIA e-reading rooms that have information related to the rule or the behind-the-scenes process of creating the rule.  Most helpful would be something like Labor/MSHA's  single source  pages, which group frequently FOIAed documents related to a single subject.  Underlying docs, like reg impact analyses, are sometimes strewn throughout websites and not gathered in some of the e-docket/commenting sites I have visited (although I typically rely on other sources, so I may simply have failed to see them if they are readily available there).  Add advanced searching tools, like Westlaw's search features (same sentence, same paragraph, within X words, and nested combinations of those) and field searches for more targeted searching (like Westlaw's SY,DI searching, which helps to ensure search results that are really focused on the search subject and do not randomly have search terms in a particular combination)
I am very concerned about the imbalance in the ability of trade associations/corporations vs. public-interest groups to gather information about supporters and opponents of an issue. Electronic commenting is easy, but personal contact information is too easily  available to the public. I don't think people intend for their contact information to be instantly available anywhere in the world when they send a comment to the government.  For this reason, I strongly suggest that the option be made available to all commenters that their personal info (name, contact information) be blanked out when it is electronically posted. The  full information would be available in the offices of the government agency.  The option should be at both the top of every docket, and on the dockets management pages through which a prospective commenter  has to scroll through.  I personally feel this lack of privacy is a barrier to commenting.
The agency should ensure that all scientific, technical, and other supporting documents that are used or relied upon by the agency are including in the electronic docket.
Needs better search capability, generally, and within a specific docket. Needs better navigation capabilities within a specific docket. Needs clearer instructions regarding attachments, e.g., how to and what format. Needs to allow download of multiple documents in a single step.
The OSHA e-docket system does not have any capability to search the substance of submissions.  The search of any documents in an individual docket is limited to who (person, organization) sent the submission.  I would like to see greater search and indexing capability.
Easier and more accurate confirmation processes.
enable the review of comments by an electronic tool that will scann multiple comments and rank them according to a key word priority
More interactive
The agency should provide a user-friendly, non-technical explanation of the purpose and functioning of the docket/commenting system.  Individual comments should be labelled more clearly -- perhaps using a code inserted by the commenter or the agency.
 
I'd prefer a proactive system, all information is available and subject to comments but not all sources are  labelled so you have to waste a long time in reading information that doesn't match your interests.
Define the sources of dockets before you ask questions about them.  No space to add that in addition to being a concerned citizen,  activist, etc., I am other: elder, grandmother replying for  future generations.  When will we stop consuming their water?
Would like it configured to accept only good rule proposals and reject dumb rule proposals.
would like to be able to manipulate data...i.e. sort, compartmentalize, batch, find, etc.
More prominent links to other comments and supporting and related documents.
Still could use some improvements that would make it more user-friendly, and that would provide current status information on the rulemaking.
Date range searches Separate one-page non-substantive comments from those that are more substantive.
That the proposed rule was summarized in non-legalize making it more understandable.
Sometimes they ask biased questions -- questions should always be impartial and neutral.
Easier searching and a summary search response.   Current electronic systems assume that everyone understand how that agency identified information
searchability
This survey has been an educational tool for me, as I was unaware of  this tool to be part of government/policy work.  I would like to learn more about this.
accept/post other than text files
I lack sufficient computer experience and experience great difficulty in finding documents/dockets that are relevant to our issues due to disabilities. As a matter of fact just using this box to comment to your survey is causing problems because I hit the enter key and believe but am not certain that I lost the paragraph or two that I entered prior to that action. I have greater difficulty finding dockets/items that are relevant to myself and the issues that our organization focuses on than I ever had in the era of paper documents.
The rulemaking process is far from transparent and the agencies are not accountable. Rulemaking success should be measured in terms of compliance with democratic norms. Interest groups and powerful lobbies have the heads up on actions. The are invited to regneg sessions at the exclusion of the public. The public is not informed on the process and therefore not able to participate effectively. It is also clear that there is little substantive change from the proposed rule to the final rule. This suggests to me a fait accompli process where the influence and the real decision is made before the public is really actively involved. Finally, finding information in the FR and CFR via websites is difficult and unreliable.
A different, more intuitive way of visualizing the process.
I think that more dockets should have tutorials and explanations about the process. In addition, explanatory web sites like the organic foods and roadless rulemaking sites  should be standard.  More of the background materials supporting the proposal should be put on the Web. Federal Register fees limit the size of preamble texts and the Web can solve this. Analytical modeling tools could also be mounted allowing parties to explore specific outcomes.  Rule compliance wizards would also be helpful. They would allowed regulated interests see how the rule would impact them.  Dockets should have listserv features like DOT's that alert parties that  a new comment has been filed.
A meta-tag function should be added to the Web form for submitting comments that allows each commenter to categorize his/her comment according to a number of categories (e.g. support, modify, oppose, etc). These self-assigned meta-tags would allow users of the docket system and agency staff to sort large numbers of comments.
What I'd like to change re this one: on previous question there is no way to say a) this may increase access for some citizens but decrease it for others; b) it's probably an improvement over paper comments for some people and not others; c) the question re preventing having to go to DC to read paper versions in unanswerable because I wouldn't have gone to DC but have ordered proposals in print by mail. I don't know if there's another chance to enter comments so will say here that I'm very concerned that overreliance on electronics will cut out those who don't use them for religious or financial reasons.
That more grassroots people be aware of being able to participate
We don't have such a system in rulemaking in the Montana state  government.
Haven't used it yet ... usually send comments via e-mail, mail and fax.
I reviewed individual comments, and am unaware of whether I could have searched by terms to identify comments addressing issues of particular interest to me.
I am not familiar enough with the system to respond to this from a commenter's point of view.
Lousy search interface
Still not able to determine when a particular rule is open for comments.  Since agencies cannot respond to inquiries in any detail while a proposed rule is being vetted by OMB, there is a gap until the rule is finished at OMB and then appears in the CFR.  also not easily to see other people's comments to revise own comments or refute those made by others.
I never used one.  Sounds like an excellent idea though.
First of all, no one in my office has been able to get  Regulations.gov to work effectively: every time we click to make a comment it launches a new browser window or returns to the home page (we have reported this problem many times, with no response). Also, the searching is really weak and doesn't handle phrases well at all.  Once it is working, I strongly believe that a central point for  reviewing proposed rules, reviewing comments, making comments, and reviewing responses is critical to the success of eRulemaking in general, and Regulations.gov is certainly headed in the right direction, even if the first step has been what appears to be an expensive, abyssmal disappointment. Specifically, I would like  1. to be notified via e-mail when proposed rules are posted that match my interests and/or needs (i.e., profile- keyword-based automatic notifications)  2. to be able to find all of the rules I am looking for from a single search interface, regardless of source, and to be able to browse proposed rules by subject area, not just agency  3. to be able to make comments at the paragraph or section level, so I don't have to keep track of citations and references when I comment (e.g.,  the requirement is paragraph four is unacceptable...   4. to be able to quickly and easily review other comments at the section or paragraph level, and to make comments on those comments at that level  5. to be able to categorize my comments by issue and to sort other comments by issue for review  6. to be able to respond to specific questions in a proposal using standard conventions (e.g., radio buttons, check boxes, etc.) and to  view a real-time summary of responses to questions  7. to easily review the agency's responses to specific comments at the section or paragraph level and/or how specific issues were addressed by the agency  8. to be automatically registered for notifications  of updates, changes, etc., once I have made a comment on a proposed rule, including when (and if) an agency posts a specific response to my comment
make it run faster better organize comments by topics
Make it simpler to comment and retrieve others comments.  The system I used seemed unnecessarily complex and I had to try twice before I could submit my comment.  Also, there should be a feature that allows the  submitter to receive an acknowledgement the the agency had received and considered my comment.  The CPR still said that the only way to receive an acknowledgement was to make a submission in writing and submit a postage paid, self-addressed postcard.  If the government is going to accept e-comments, it should be able to e-acknowledge them.
Easier & more direct navigation to the substance of the proposed rule.
Quicker posting of comments
E-docket does not, but should, present the date received (or I couldn't find it) only the date of the document. I don't know how the index info gets into the database but typos have made it impossible to search on key words I would expect to be able to search on.
Index/summary of submitted comments -- one short sentence on the viewpoint expressed or group stance.
In my experience, on the day that comments are due, I sometimes can't get to the docket.  I assume that's because many people are trying to file their comments at the same time. So I think that EPA needs to increase its server capacity to handle all of the traffic on the deadline day.
EPA does a magnificent job with its e-docket.  All federal  departments and agencies should follow suit.  My experience with the Fish and Wildlife Service, in contrast was very frustrating, as they have no electronic docket.  I've been spoiled by EPA.  Docket should immediately provide on screen and/or by email to  the submitter a  receipt  for comments submitted via the internet, showing date and time of receipt.  Since the document(s) thus  submitted are not immediately posted to to the docket, a receipt  would give the submitter assurance that comments have been  received by the the applicable deadline.  I would encourage putting contents of older dockets into the  e-docket system, particularly those that have a significant level of stakeholder interest.  For docket listings that exceed one computer screen, by default show how many screens are needed to see the entire docket listing, with links I can click on to go to any one page of that listing without having to page through the entire list to get to the end. For example,  Docs 1-30, 31-60, 61-90, etc.
Navigation around the site should be rapid and intuitive. List most recent postings first. The Agency should be committed to the system and keep it up to date.
Make sure that the items listed in the edocket are really  accessible. A recent edocket which was supposed to contain a RED  document for a chemical and the docket indicated that the document  was not posted. EPA argued that it was, but there were apparent  technical problems. I was able to access the docket a few days  before the comment period closed.
I had difficulty sending items in PDF format because I do not have Adobe write capability, so my original comments did not register on the system.  The manager of the process allowed me to send her a Word file that she then saved as a PDF file.  The person I dealt with was very helpful.  You should allow people to submit comments as either a PDF file or a Word file.  The directions are confusing and should be clarified.
Electronic dockets will likely be subject to mass responses from activists that may swamp the ability of the system to cope.  This could be especially true where generic comments are prepared by a group and circulated to its membership.  Multiple copies might be sent by single individual to either influence the count of pro or con talleys or to attempt to exclude or discourage other opinions.  The e-Docket may haave no capability to handle the number of responses.
Indicate clearly the last update made to the comments  contained in the docket and the frequency of updates for  new comments.
In scrolling thru the docket to review the comments submitted, I could not pick up from the last comment I reviewed as more comments were added without going thru all the other existing comments.  That is, I could not jump to the last comment by number, and then pick up scrolling thru new comments added in numerical (chronological) order.  Lack of this capability restricted my ability to review the docket.  I could search for particular documents that I new existed, but I could not review documents that I did not know existed as the docket grew large.
information posted more quickly
The creation of an overview document that captures in a good summary the action that is being taken is one of the most important aspects of any e-comment system.
Better information on the people who comment. Those who participate would be asked to fill a form giving data on themselves, their relationship to the project, any group, organization, business or institution with which they are affiliated and which has a bearing on the project, the areas in which they claimed expertise, and the basis for those claims. That information in turn should go into a database that allows easy sorting and searching. Oftentimes, when one reads comments, it is not clear who is speaking on whose behalf. This information should be requested in a friendly format designed to respect people's sense of privacy, and should not be readily downloadable.
Allow multiple attachments per entry.
Create an interactive comment system that would focus comments on specific issues and organize comments by issue.  The intent would be to minimize irrelevant comments and help organize and focus comments.  This could be by funneling commenters by subject, asking specific questions (similar to this survey), or a combination.  This would probably need to be supplemented by a capability of free form comments also, to make sure no relevant comments are discouraged.  This would save time for agencies and help them focus on relevant comments.
Used  it twice.  Well, I TRIED. (1) first attempt involved the FCC.  At deadline, the system JUST DID NOT WORK.  Period.  Fortunately, I had a friend who was a Commissioner!  Sent the comment directly to her e-mail for filing in the docket.  If I didn't have that capacity I would have been very, very, very upset, as I had invested a whole lot of time in the process (and had some comments that raised issues involving technology that the agency had not considered). (2) second time, FDA.  The electronic commenting system would only let you answer THEIR particular questions.  It tried to  steer  you in a certain way that had absolutely nothing to do with the issues I wanted to raise.  So I landed up putting what I wanted to say within a framework of a certain set of questions -- to which I was being quite nonresponsive.  There was no  free response  option in the electronic docket (which, of course, I screamed about, loud and clear ...).  So, what to change? (1) Make sure there is technical support available -- in the form of an e-mail address (to which, in the worst case, you could always attach your document), and (2) Don't assume the public commentors need to be led down a  path,  but rather leave open the possibility of people saying ... what they want to say.  That is, after all, the point of comments...
I'd like it to have bookmarkable, human-readable URLs that made it easier to find a particular proceeding and/or specific comments within that proceeding, as well as to refer others there.
Add a cross reference to recent related rulemaking or pr notices.
Consider providing auto-feedback email regarding updates made to  specific dockets to people who have previously submitted comments -- particularly if there were other comments offered in support  of or rebuttal to their comments.   Also:  Consider allowing 'intelligent' searches through entire population of comments, based on specific search parameters or subject material.  This would focus searches on specific subject matter, rather than having to research entire comments database for specific subject material of prime interest to commenter.

This research project was initiated during the fall 1999 semester and was made possible with the following grants from the National Science Foundation: IIS-0429293 “Collaborative Research: Language Processing Technology for Electronic Rulemaking,“ EIA-00328914 “SGER COLLABORATIVE: A Testbed for eRulemaking Data,” SES-0322662 “Democracy and E-Rulemaking: Comparing Traditional vs. Electronic Comment from a Discursive Democratic Framework,“ and EIA-0089892 “SGER: Citizen Agenda-Setting in the Regulatory Process: Electronic Collection and Synthesis of Public Commentary.”

Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the National Science Foundation.