How about:

on !@*:JOIN:#chan: {
  if (!$hfind(flyby,$+(*,$wildsite,*),1,w)) hadd -mu10 flyby $calc($hget(flyby,0).item + 1) $wildsite
}
on !@*:PART:#chan: {
  if ($hfind(flyby,$+(*,$nick,*),1,w).data) {
    ban -u600 $chan $nick
  }
}

Now, it can be made much better, to work with several channels etc etc, i just made one to match up with what u did.