The low replication rates observed in psychology imply either that the psychological literature is filled with false positive results or that psychological effects vary considerably even when procedures and methods of primary studies are recreated as closely as possible. The latter possibility has recently sparked much debate about heterogeneity in psychological effects and provoked heightened interest in meta-analytical estimates of such heterogeneity. Several authors have argued from a meta-theoretical perspective that heterogeneity has to be expected due to the typically large number of relevant but unknown (and, hence, uncontrolled) factors and moderators. In contrast, direct replications are clearly designed and conducted under the assumption that faithfully reinstating the procedures of original studies should suffice to replicate their findings. In parallel to these debates, several large scale projects (Many Labs, Registered Replication Reports) provided data that, for the first time, allow for an empirical assessment of heterogeneity across a sizeable number of directly replicated psychological effects. In the first part of this talk, I will give an overview of the current debate around heterogeneity and summarize the results of previous meta-analytical heterogeneity estimates in replication projects. In the second part, I will discuss several problems with these estimates stemming from both the methods applied in conducting replications and in meta-analyzing their results. A central theme of this discussion is that there is little reason to expect homogeneity in psychological replications even when moderators are absent. In turn, heterogeneity does not imply the existence of moderator effects but may simply be due to several methodological artifacts.