OVERVIEW
The Journal of Systems Research (JSys) is a diamond open-access (no fees for authors or readers) journal that seeks to publish research that moves the field forward and will be of interest to the systems community. JSys emphasizes rigorous research that improves the state-of-the-art, even if the increment is modest. JSys requires that papers that put forth a new tool, benchmark, or software solution undergo Artifact Evaluation and make their artifact publicly available.
JSys currently seeks submissions in seven broad areas:
* Configuration Management for Systems
* Streaming Systems
* Distributed Consensus
* Systems for ML and ML for systems
* Serverless Systems
* Real-time and Cyber-physical Systems
* Data Science and Reproducibility
JSys categorizes papers into four types, each with their own publicly-available acceptance criteria:
- Systemization of Knowledge (SoK) papers
- Tools/benchmark papers
- Problem papers
- Solution papers
SUBMITTING A PAPER
Papers should be submitted electronically as a PDF. JSys uses OpenReview for reviewing the papers. Please be advised that papers will be publicly available upon submission.
FORMATTING INSTRUCTIONS
All text and figures must fit within a 7” x 9” text block, centered on the page, using two columns with .33 inches of separation. The paper should be typeset using a 10-point font (Times New Roman or similar) with a 12-point (single-spaced) leading.
PAGE LIMITS
There are no page limits. However, we strongly encourage authors to make an effort to make the paper as brief as possible: reviewer attention is limited, and a compelling case for the paper must be made in the first 12-14 pages.
TEMPLATE
We provide a template at https://github.com/jsysresearch/template. Authors can do the following to get the template and build the sample PDF:
git clone https://github.com/jsysresearch/template.git
make
DOUBLE BLIND
JSys reviews papers in a double-blind fashion: the reviewers do not know
who the authors are, and the authors do not know who the reviewers are (until acceptance). The authors must make a good-faith effort to ensure the submitted document cannot identify the authors or their institution. This means that if they are building on their own prior work, it should be referred to in the third person (“The paper by Brown et al t..” vs “Our previous paper..”).
ARXIV
JSys allows submission of papers already uploaded on Arxiv. However, the title of the JSys submission must be different from the Arxiv submission, and the system name used in the two papers must be different. The authors should make a good-faith effort to preserve the double-blind nature of the JSys submission. Please inform the Editor-in-Chief and Area Chairs if there is a previous Arxiv submission; this helps if reviewers unintentionally find the Arxiv paper.
SOCIAL MEDIA
JSys does not place a restriction on advertising your work on social media. However, do not link to the OpenReview submission; doing so breaks double blind. For example, tweeting “We have done some work on FooBar, here’s the Arxiv paper” is fine. Tweeting “We have just submitted our work on FooBar to JSys! We are submission #42” is not fine.
OPEN SOURCE
JSys encourages authors to make their code, data, and other artifacts open source. However, if the open source repository is not anonymized, it should not be linked from the paper submission. Making non-anonymized artifacts publicly available is fine as long as the system name in the public repo is different from the JSys submission. A short-hand for remembering this is that the open-source rules are similar to how Arxiv papers are handled.
COPYRIGHT AND ORIGINALITY OF THE WORK
All work submitted to JSys must be original work. The authors must have the copyright for the submitted work. Simultaneous submissions are not allowed: any work submitted to JSys must not be under consideration for another workshop, conference, or journal. Upon submission, the work will be publicly viewable on OpenReview: authors requiring secrecy of their submitted work should take this into account.
CONFLICTS
When submitting your paper to JSys, you must provide information about conflicts with reviewers on the editorial board. A reviewer is a conflict if any of the following three circumstances applies:
- The reviewer is your PhD or post-doc advisor.
- You are currently employed at the same institution, have been previously employed at the same institution within the pasttwo years (not counting concluded internships), or are going to begin employment at the same institution during the review period.
- You have collaborated with the reviewer on a publication, grant, or other academic projects in the past two years.
- You must not improperly identify a reviewer as a conflict if none of these three circumstances applies -- your paper may be rejected in that case. The following are not valid conflicts:
- They have reviewed the work before
- You have discussed the work with them
- They are your personal friend
The Editor-in-Chief and Area Chairs will review paper conflicts to ensure the integrity of the reviewing process, adding or removing conflicts if necessary. If you have any questions about conflicts, please contact the editor-in-chief and area chairs.
ARTIFACT EVALUATION
JSys submissions for the Solution or Tools/Benchmark category must be accompanied by an Artifact. The Artifact will be evaluated by our Artifact Evaluation Board as to whether it can be used by a third-party (without author involvement), and whether the results in the paper can be reproduced using the artifact. Note that Artifact Evaluation will proceed in parallel with the paper evaluation; to be accepted, Solution and Tools/Benchmark papers must be accepted by both boards. Papers accepted by the Editorial Board that fail Artifact Evaluation will be given a revise decision, and will have three months to ensure their Artifact passes Artifact Evaluation.
DEADLINES
JSys has multiple deadlines each year. The deadlines for 2021 are:
March 1, 2021
May 1, 2021
August 1, 2021
November 1, 2021
Submitting by these deadlines will ensure reviews are available in 1.5 months. For example, if authors submit by March 1st, they will get initial reviews back by April 15th.
REVISIONS AND REVIEW PROCESS
Possible decisions for a paper submitted to JSys are:
● Accept with shepherding
● Revise
● Reject
Acceptance with shepherding implies the paper can be accepted with minor edits that can be completed within one month. Thus, authors can submit on February 1st, get an acceptance decision by March 15th, and submit the final manuscript by April 15th. The final manuscript will be made immediately available online.
A revision decision implies the paper needs work that will take more than one month. The authors can submit a revised version anytime in the next three months. A revision submitted on the 1st of a month will get a response by the 1st of the next month. If authors submitted on March 1st, and got a Revise decision on April 15th, they can submit by June 15th and get a final decision by July 15th. A revision is a contract between the authors and the reviewers: as long as the authors complete the requested changes in good faith by the deadline, the paper will be accepted after a quick review. However, the editors reserve the right to reject the paper if additional evidence indicates that the central hypothesis or result of the paper no longer holds.
For the revision, the authors will submit a PDF containing two parts:
A revision plan explaining the changes made, and how they address reviewer paper
The revised manuscript with changes highlighted for easy reviewing
A rejection implies that the paper needs more than three months of work to get to acceptable quality. The authors cannot submit the same paper for one year to JSys. A rejection decision usually stems from a fundamental mismatch between the work and the journal such as the work is too narrow or out-of-scope. Rejection decisions imply that even with more experiments or revised writing (that can happen in three months), the paper would not be acceptable for the journal. We hope to reduce the number of rejection decisions by being clear about what would be acceptable for the journal.
CONTEMPORARY WORK
Any peer-reviewed work that was published (PDF available on the publisher’s website) less than two months prior to the initial submission is considered contemporary with the submission. Reviewers will not penalize the submission for not comparing with contemporary work. However, reviewers may ask for a qualitative comparison to be added in the final version of the submitted work.
FEEDBACK ON REVIEWS
The three reviews will be graded A/B/C by the authors in terms of helpfulness. These grades are only known to the editor and provide useful feedback about the reviewers. The scoring A means a constructive review, B means an average review, and C means a terrible review (i.e. the reviewer didn't understand anything, made a lot of mistakes, was unnecessarily aggressive, etc). Reviewers with a history of poor reviews will be removed from the editorial board.
RETURN WITHOUT REVIEW
When a paper to submitted to JSys, the Editor-in-Chief and the Area Chairs will try to find expert reviewers for the paper. If they are unsuccessful, or if the relevant Editorial Board members already have been assigned their maximum load for the deadline, the paper will be returned to the authors without review (usually within a week or so). We believe this is a better outcome than overloading the reviewers and lowering the quality of reviews for everyone who submitted for that deadline. Papers re-submitted due to a return will get preference in the next review cycle. We aim to tailor the editorial board size and expertise to prevent this situation from happening as much as possible.