I don't have a guaranteed solution for you as the GPU card is old and is not officially qualified.
Anyway, to explain the pciHole.start, pciHole.end, pciHole.size they are related to the PCI hole in a VM with a BIOS environment.
So the PCI hole would be under the 4GB address range in a VM with BIOS as its virtual firmware. The start and end is to specify the starting and ending address.
So if you specify
pciHole.start = "2560"
pciHole.end = "3072"
You have a PCI hole size of 512MB as 3072 - 2560 = 512.
The PCI hole is the reason why most (if not all) operating systems (EDIT: that run on hardware that traces its design roots to the original IBM PC in 1981) will not report the installed memory that one might expect. For example, you might have a machine (virtual/physical) with 8GB RAM but the OS reports only 7.8GB RAM installed.
So you could try enlarging the PCI hole (assuming you are using BIOS as the virtual firmware for the VM) to accommodate the 256MB???? that the GE7300LE has. The larger the hole the less the OS has for its own use on the address range below 4GB.
Alternatively, you could also create a VM that uses EFI instead of BIOS for virtual firmware. The PCIe device address then can go beyond the 4GB address range. You need a 64-bit address so you need these entries for a VM with EFI as its virtual firmware. The size has to be a power of 2 so I suppose 1GB is the minimum since 2 to the power of 0 is 1.
pciPassthru.use64bitMMIO="TRUE"
pciPassthru.64bitMMIOSizeGB="2"
You can look at this KB if you haven't yet