We develop an atomistic, nearest-neighbor sp(3)s* tight-binding Hamiltonian to investigate the electronic structure of dilute bismide alloys of GaP and GaAs. Using this model, we calculate that the incorporation of dilute concentrations of Bi in GaP introduces Bi-related defect states in the band gap, which interact with the host matrix valence band edge via a Bi composition dependent band anticrossing (BAC) interaction. By extending this analysis to GaBixAs1-x, we demonstrate that the observed strong variation of the band gap (E-g) and spin-orbit-splitting energy (Delta(SO)) with Bi composition can be well explained in terms of a BAC interaction between the extended states of the GaAs valence band edge and highly localized Bi-related defect states lying in the valence band, with the change in E-g also having a significant contribution from a conventional alloy reduction in the conduction band edge energy. Our calculated values of E-g and Delta(SO) are in good agreement with experiment throughout the investigated composition range (x <= 13%). In particular, our calculations reproduce the experimentally observed crossover to an E-g < Delta(SO) regime at approximately 10.5% Bi composition in bulk GaBixAs1-x. Recent x-ray spectroscopy measurements have indicated the presence of Bi pairs and clusters even for Bi compositions as low as 2%. We include a systematic study of different Bi nearest-neighbor environments in the alloy to achieve a quantitative understanding of the effect of Bi pairing and clustering on the GaBixAs1-x electronic structure.