We present a sample of $i{775}$-dropout candidates identified in five Hubble Advanced Camera for Surveys fields centered on Sloan Digital Sky Survey QSOs at redshift $zsim 6$. Our fields are as deep as the Great Observatory Origins Deep Survey (GOODS) ACS images which are used as a reference field sample. We find them to be overdense in two fields, underdense in two fields, and as dense as the average density of GOODS in one field. The two excess fields show significantly different color distributions from that of GOODS at the 99% confidence level, strengthening the idea that the excess objects are indeed associated with the QSO. The distribution of $i{775}$-dropout counts in the five fields is broader than that derived from GOODS at the 80% to 96% confidence level, depending on which selection criteria were adopted to identify $i_{775}$-dropouts; its width cannot be explained by cosmic variance alone. Thus, QSOs seem to affect their environments in complex ways. We suggest the picture where the highest redshift QSOs are located in very massive overdensities and are therefore surrounded by an overdensity of lower mass halos. Radiative feedback by the QSO can in some cases prevent halos from becoming galaxies, thereby generating in extreme cases an underdensity of galaxies. The presence of both enhancement and suppression is compatible with the expected differences between lines of sight at the end of reionization as the presence of residual diffuse neutral hydrogen would provide young galaxies with shielding from the radiative effects of the QSO.