It has been submitted that the implementation of technological innovations in adjudicatory processes reduces error and improves consistency in legal decision-making, is cost and time efficient and benefits access to justice. An issue that has received little exposure is that of the impact of automation on the rights adjudicated. In other words: does (partly) automated dispute resolution lead to results equal or approximate to those obtained in procedures conducted under the auspices of a human actor (mediator, conciliator, arbitrator or State-appointed judge) only?This paper seeks to investigate this issue from three perspectives: computer science, (cognitive and social) psychology and experiences with automation in the context of US administrative law. It applies the insights gained from these fields in a case study in the field of European Consumer Sales Law (“ECSL”). This case study examines whether the use of automation in dispute resolution over rights defined under ECSL affects the vindication of such rights. It is submitted that the results of this assessment give rise to concerns, amongst others for doubts as to the compatibility of automated decision-making and ECSL, market pressures incentivizing unbalanced algorithms (automation bias) and perils of human interaction with automation (decision bias, complacency and monitoring failures) more generally. These concerns are exacerbated by a lack of (judicial and regulatory) supervision of automated decision-making systems on the EU and national levels.This analysis suggests an alternative approach towards regulating automation in (online) dispute resolution. This approach focuses not on developing codes of conduct directed towards developers of expert systems or ADR professionals employing such systems. Rather, it proposes that regulation in the field should address the human-system dynamic in automation by incentivizing the development of transparent systems capable of internalizing trust in automated decision-making.