Large arrays in VB6
Hi everybody,
I have a very stange problem about large arrays in VB6. I have to write a program that deals with an array containing several hundret MB. Having installed 2GB of RAM on an WinXP SP2 machine this shouldn't be a problem. However I always get an runtime error 7 'Out of memory' whenever the array gets bigger than a certain limit. The interesting thing is, the limit itself is not always the same: sometimes I can do 250 MB, sometimes up to 800 MB.
To trace down the problem I wrote some very simple lines of test-code:
Private Sub Command1_Click()
Dim arr() As Byte
ReDim arr(300, 1023, 1023) As Byte
MsgBox "Works!"
End Sub
I read already a lot of information about this issue, so I know, I must not use Forms 2.0 controls, etc. I saw also the thread form 2002 about this issue, where somebody suggested, it could be related with memory fragmentation.
However, can somebody please provide me more information regarding this?
And more important, does anybody have an idea on how to write a program that can open an array of 1 GB in memory, regardless of possible memory fragmentation (providing that the machine has at least 2 GB of physical memory installed)?
Thank a lot in advance for any suggestions!
Best Regards,
Tom
[1351 byte] By [
tomscholz] at [2007-11-11 10:14:24]
# 1 Re: Large arrays in VB6
I think it has relation with the relocation part, as the redim statment is a runtime operation which make the wanted size in the memory at run-time, otherthan the program normal length which could be determind at just start running or at debug time too...
to explain this; when u try to load a program, the os gives the loader a suitable memory adress which under it there is a free space to load in it all the progrm in the memory, but the redim statment is loaded as an order and its size has not been considered, so when it goes here it depends on this moment wether your ram has free space enought or not, and it varies some time u have 800 MB availble space, some time 300 ad some time larger and lower ... otherwise u may use not the redim statment and use it like that : dim a(size) as byte , to tell the program the required size before loading and before running it will tell if it can or not , and here u can reduce the space used by other programmes till u could load your program in the memory .
Amahdy at 2007-11-11 17:22:56 >
# 2 Re: Large arrays in VB6
even if the pc have 4G of RAM, does not mean you can use all of it. In a multi-user, multi-tasking OS like XP you have a lot of programs running at the same time (open the task manager and select Memory Usage to see how much memory each program is using) When there is not enough memory on RAM, the memory is taken from the disk (it is said that the application is "swapping" memory, or that it runs in "virtual" memory) An application that use virtual memory runs much slower than a small app that does not need to. And even that, the "memory" space of an application is limited, you can't claim more memory than that.
So, there is not really an answer to your question, the amount of memory you can allocate depends on how much memory your app is already using. The only way to handle this situation is to catch the error and take the most convenient action
Marco
mstraf at 2007-11-11 17:23:56 >
# 3 Re: Large arrays in VB6
Just curious... What is the purpose of the program and why do you need to load it all into memory at once ? Could it not be accessed from as a file in say 4k sized blocks ?
cody3 at 2007-11-11 17:25:02 >
# 4 Re: Large arrays in VB6
Thank you all for your comments!
Actually it does not make a difference whether I use the redim statement or declaring the array directly with dim. The only difference is, with redim I get a treatable runtime error, whereas with dim the program would refuse to start.
What I forgot to mention is that of course I checked with the taskmanager that there's enough memory available. In my tests I had always between 300-400 MB memory used, which leaves still 1.6 - 1.7 GB of free RAM. In my opinion more than enough to create an array of "only" 1 GB memory.
@cody3:
The program deals with a distance-matrix. Given that a country has 9000 zip codes, I need an "arr(9000,9000,1) as integer" array, storing for every connection the distance and driving-time. Btw, this array consumes just over 300 MB and I'm not able to dimension it!
Storing all this in a database or textfile (actually I have it already in a SQL database) is not an option because of simple performance issues. During runtime I have to access millions of random distances and a database is just too slow for this (at least the server I have available). There's nothing faster than an array!
Tom
# 5 Re: Large arrays in VB6
Yea that's what I told u about it, u will know if the program will be able to work or not before running it, and in this case all your ram free memory could not hold this array other than the case of redim wihch creat this at run-time and may be able or may not;
look the dim statment make the array size a member of the program size and ask the OS for a place in the memory that could hold it all, and sometimes the response is out of memory, but in the case of redim the program ask only for a place for the size of the statment "redim" without the size of its array, and when at run-time you start executing redim it try to search for this free memory in the ram , if it could not find it it pop up the out of memory error msg.. I told u to use dim to just know before starting if u can or not, if not so u need to reduce the running programmes till the program could execute normaly , but remember that the Ram hardware itself has limitation in its structure , which is format of sections of different size to hold different types of data and programmes and maybe your ram structure hasn't a sectore type that could hold a complete 1 gb at one time, [and this is maybe at the case of two rams each of 1 GB] ... so whenever u have a free memory u will not be able to load this size in one sector, and u may need to divide this work for different arrays and assemble them in your program logic only , but also in this case , this method is not safe at all and u may not use it forever and will got too many problems so I advice u to not use this method and use files ... more time is better than more problems .
Amahdy at 2007-11-11 17:26:55 >
# 6 Re: Large arrays in VB6
You are talking about limitations of the RAM hardware:
... that the Ram hardware itself has limitation in its structure , which is format of sections of different size to hold different types of data and programmes and maybe your ram structure hasn't a sectore type that could hold a complete 1 gb at one time, [and this is maybe at the case of two rams each of 1 GB] ...
Do you have a link, or a source of further information about this, because I never heard about it and I'm curious to know.
The only thing I always read in the Microsoft documentation is that arrays are only limited by the available memory.
But there seems to be a different limitation.
Thanks!
Tom
# 7 Re: Large arrays in VB6
Ok maybe it sould not be "Ram hardware" but "OS using Ram hardware" as each operating system has a method to generate those sectores in the Ram, and this is because it is the responsable of organizing the Ram , getting memory adress required for progs to ran and all this mangments .. this could be found in any article that explain how os organize memory and make sections in it.
Amahdy at 2007-11-11 17:29:00 >
