Citadel is expanding its quantitative investment business and broadening its talent search to include AI labs, as the hedge fund competes with Silicon Valley for researchers.