Diese Seite mit anderen teilen ...

Informationen zum Thema:
Forum:
WinDev Forum
Beiträge im Thema:
25
Erster Beitrag:
vor 4 Jahren, 12 Monaten
Letzter Beitrag:
vor 4 Jahren, 11 Monaten
Beteiligte Autoren:
Frans, JP, Al, Stefan Bentvelsen, Michel GARCIA.pcs.crosspost, Sylvain ROBERT ( Dat).pcs.crosspost, Piet van Zanten, Fabrice Harari, GuenterP, stefan.kern, Bart VDE

More damaged indexes

Startbeitrag von Frans am 25.04.2013 10:18

Hello to you all,

The last year my customers get more damaged indexes as usual.
There has nothing been changed in this part off the pragram.
My thoughts:
- I changed from W16 to W17??
- More customers use W7 (before XP)
- I am almost certain that W8 gives more problems
- Energiecontrol?? (computer goes to sleep and also the screen and HD)

What did I do: Adviced everybody to put energiecontrol out.
What could I do: After everytime they don't close properly the computer a check on the indexes. But this is very time cunsuming!

Any other thoughts?
Thanks in advance.

Antworten:

Hello Frans

My experience with HyperFile leads me to say that it is probably hardware related. I have been using Windev since v5.5 and the hyper file and its indexes are very robust. With the exception of one client who has bad wireless connections between the server and workstations in the workshop, it would be a couple of years or so since I had to reindex files at a client site. They are much tougher than the dbf files I was using in my Foxpro apps.

You could check the setting on the network cards, make sure they do not turn themselves off to save power.

I have found in Windows7, on a long running process involving files, it sometimes thinks the app has stopped working and shows a message to the user telling them the program has stopped. This never happened in XP.

There is a setting in the Files section of the project to turn on automatic reindexing. If you have this turned on, the indexes will automatically reindex if there is a problem.

I don't have any clients running V17 software as I couldn't get it to work properly so all my clients are running V16 software, but I can't see that making any difference in the final app.

Regards
Al

von Al - am 25.04.2013 10:47
Hello Al,

Thanks for your answer.

All the problems come from single-user applications. So the network-card isn't an issue.
I use Hsecurity(*,2) and on some occasion HFlush()

When my program closes a stop.txt is writen in the directory. When the program starts again it looks if the file is there. If it isn't then a warning is displayed.
The users always say that they properly close the application. So there should not be any warning?!
I don't now what the users do. So I thought the energiecontrol is the bad guy?

von Frans - am 25.04.2013 11:56
Maybe you could try doing a disc check on the customer machine ?
I remember having the same problem with one customer and it was due to harddisc failure (bad sectors)

von Bart VDE - am 25.04.2013 12:41
Hi Bart,

Didn't think about that! To focussed on programming.
Wil give it a try.
Thanks

von Frans - am 25.04.2013 16:49
Quote
Frans
Hello to you all,

The last year my customers get more damaged indexes as usual.
There has nothing been changed in this part off the pragram.
My thoughts:
- I changed from W16 to W17??
- More customers use W7 (before XP)
- I am almost certain that W8 gives more problems
- Energiecontrol?? (computer goes to sleep and also the screen and HD)

What did I do: Adviced everybody to put energiecontrol out.
What could I do: After everytime they don't close properly the computer a check on the indexes. But this is very time cunsuming!

Any other thoughts?
Thanks in advance.


Hi Frans, perhaps these are short power losses of the mains. Or spikes on the mains. Use at least filters like http://www.pearl.at/at-a-PX5893-5452.shtml for protection of PCs. Or use USVs like http://www.pearl.at/at-a-NC5283-5452.shtml?query=usv just as an example. Next, you can switch off the caching of Windows which will 'flush' each database change (data + index!) to disk. Makes PCs a lot slower but much safer. Kind regards, Guenter

von GuenterP - am 25.04.2013 17:29
Hello Guenter,

Thanks for your answer. Do you mean HFlush?
I also got the damaged indexes on laptops. I thought they had a sort of ups?

The files settings in the description of the project is set to 'Automatic reindexing' but i never ever had a warning that an index was damged and/or repaired. And the 'indexing window' is set to visible.
I think I don't understand the automatic reindexing. To me it seems that it is not automatic.

von Frans - am 25.04.2013 18:16
Hi Frans,

you are using HF Classic?
There is a problem with file based databases on Windows OS using the SMB2 network protokol, as WIN7 and Server 2008. Ask google for "opportunistic locking".
I have an old VFP app which runs beautiful using XP as client and win2003 as server, and it is a nightmare using win7 and Server 2008. Corrupt indexes every week.

The new windows systems do not really support file based databases. Try to switch to HFCS or any other DB Server.

Best Regards

Stefan.

von stefan.kern - am 25.04.2013 22:18
Hello Stefan,

Thank you for your answer. I think this artice in the help is also interesting:
http://doc.windev.com/en-US/?3044372&productversion=XXA150056M

If I understand it correctly then this is only a problem in a network. Or am I wrong?
But the majority of the corrupted indexes is in single-user mode.

von Frans - am 26.04.2013 07:32
Hi Stefan,

The problem with file based systems like VFP and WinDev Classic HyperFile was to do with SMB2 used in Windows Server. There are reports that this issue seems to be fixed now with the newer SMB3. See this link (relevant posts are towards the bottom)

http://social.technet.microsoft.com/Forums/en-US/smallbusinessserver/thread/8d8a00c1-b497-4fec-b894-40ff73e7fe42/

Excerpt from Hank Fay - well known VFP expert: The SMB2 locking issue with VFP is known to Microsoft (google or bing "msdn smb2 vfp locking" without the quotes). And the initial reports on SMB3 is that this appears to solve the problem...

von JP - am 26.04.2013 08:00
Hello

Frans has indicated that he is in single user mode so networking, oplocking, SMB protocols etc do not seem to be relevant. That could be proved simply by compiling a single user version for testing on some sites. Power would not seem to be an issue either because the problem occurs on laptops.

The problem seems to occur on a variety of equipment at multiple sites so what are the common denominators in the sites where the failures occur ?

For me, I have faith in the Hyperfile file system, so it leaves the OS or some weird bug in V17 where code executes differently to that of V16.

@ Frans

What is the exact error message your users are getting ?
Is there any difference between sites running 32bit and 64bit versions of Windows ?
Can you reinstall a V16 copy of the app at a couple of the worst affected sites and if so does this fix the problem ?


Regards
Al

von Al - am 26.04.2013 09:09
Hello ,

Thanks for the answers.
I to have faith in HF but I don't now what causes the index corruption.
My customers say that there are no errrors. But that they closed the program and started it later.
When my program closes a stop.txt is writen in the directory. When the program starts again it looks if the file is there. If it isn't then a warning is displayed.
So they didn't close the program(forgot?) or the stop.txt doesn't work well.

I have the idea that the last year there are more 64 bit machines and more laptops (with energiecontrol).
Checked the autoindexing. The automatic reindexing was always on. But I never had a screen with says that reindexing is started?

von Frans - am 26.04.2013 14:30
Hi Frans

did you check on the machine where yo have the problem id the hard drive is set to go to sleep after a while... If yes, try to unchck that option and see if it works better...

Best regards

von Fabrice Harari - am 26.04.2013 16:15
Frans,

Also check whether the user's hard disk has disk write caching enabled or not. The program might exit, "write" the stop file but the file might not be commited to disk.

von JP - am 26.04.2013 18:10
Hello JP and Fabrice,

Thanks for your answers,

Yes in the advanced section I disabled the going to sleep of the HD.
Didn't check the hd write cache. Could be an option but normally it should write the last file because I inserted a pause after the write.
Can check it again.
Thanks

von Frans - am 26.04.2013 19:53
Frans,

The pause wont necessarily write the file to disk. Instead of writing/creating a file how about updating an item/field in a HF file. Then you can use HFlush() to force the OS to write the file to disk. The item could contain a simple boolean flag to indicate whether the system was exited correctly. HTH.

von JP - am 26.04.2013 22:02
Hello Frans

I am still unclear as to the exact problem.

Is it that the "Stop.txt" file is not being written when the program closes ? In this case JP's suggestion of using an HF File is a good solution.

Is there an error message from Windev indicating the indexes are corrupt ?

Regards
Al

von Al - am 26.04.2013 23:30
Hello Jp and Al,

Thanks for your answers.
Could be that the stop.txt is not written on time.
The fact is that some users get damaged indexes. They say they get no error messages and that they always close their application.

How do they recognize the damaged index?
I have a procedure that puts a unique number (last invoice number + 1) into a field. When the index is damaged they get double invoice numbers. So the index thinks the number is unique but it isn't.
I hope I explained it clearly.

So I don´t know whats happening.
I will use an extra item in the HF log/file.

von Frans - am 28.04.2013 09:59
Hi Frans,

Quote
Frans
How do they recognize the damaged index? I have a procedure that puts a unique number (last invoice number + 1) into a field. When the index is damaged they get double invoice numbers. So the index thinks the number is unique but it isn't.


Well then you are defintely getting an index error which indicates, amongst other possibilities, disk write failures. I would do this:

1) Check the setting of disk write-caching. If on, switch this off for the moment. (Device manager -> Disk Drives -> Properties of the drive -> Policies ). Also check the related setting of write-cache buffer (same place).

2) Check the disk itself using chkdsk or other disk utility. Check the entire drive.

3) Re-create all indexes

4) Check the temporary folder for excessive number of temp files - cleanup.

5) Check anti-virus - switch off real-time scanning on your database folder. AV is virtually worthless tool these days anyway.

Q) Are there any machines running your app which do not ever have this problem? i.e. can you find cases which prove that it's not the software per se? If yes then are there any common hardware or environment features between those machines which do exhibit this problem e.g. OS, disk drive manufacturer, RAM, CPU, computer make, AV, etc.

Q) You said all problems are single user applications. But to be clear - they are not writing to an external drive in anyway; wireless, cable, USB or otherwise? i.e. all local disk?

von JP - am 28.04.2013 10:20
Hi JP,

Can you explain "AV is virtually worthless tool these days"?

Regards,
Piet

von Piet van Zanten - am 28.04.2013 10:35
Quote
Piet van Zanten
Hi JP,

Can you explain "AV is virtually worthless tool these days"?

Regards,
Piet


The debate about the effectiveness of Anti-Virus software goes up and down for many years. Some argue it’s useful, others argue not. Some say you might as well have it because it’s better than nothing. All valid points to a degree and in various applications/environments. My personal opinion is:

1) There are over 5000 new virus and malware variants being released into the wild every day (perhaps many, many more). AV software cannot possibly scan for it all let alone keep up its signature database.

2) Modern malware uses extremely powerful techniques such as polymorphic engines which change the signature.

3) Most AV programs scan for the top 100 (or whatever) current short-list of signatures (to conserve resource usage) and then has to revert to heuristic / behavioural analysis which is prone to many false-positives which most end-users have no clue about answering.

4) The AV industry is by and large built on the promotion of fear – it is a multi-billion dollar business. To see how sad the state of AV and security software is today have a look at how awful most personal firewall software is (most provide no protection at all against modern, sophisticated malware) - http://www.matousec.com/projects/proactive-security-challenge-64/results.php

5) The malware war is asymmetric - the AV industry needs to be right 100% of the time. Malware only has to be right once and you are infected.

6) The vast majority of malware infections today come via browsing malicious websites or infected websites, not via virus infected files (which is what my comment specifically refers to i.e. “anti-virus”, not the far broader “anti-malware” term which requires precautions beyond simply scanning for virus signatures)

7) You cannot protect against zero-day attacks.

8) The most powerful tool against malware infection is end-user education and common sense.

AV software might be considered an additional layer of protection but it’s debatable how effective it is in all situations particularly as a preventative security. Here is a couple of relatively recent links on the subject of AV effectiveness:

http://blogs.computerworld.com/security/20580/how-useful-antivirus-software

http://www.imperva.com/docs/HII_Assessing_the_Effectiveness_of_Antivirus_Solutions.pdf

http://www.crn.com/news/security/240008434/is-antivirus-becoming-obsolete.htm?pgno=1

http://krebsonsecurity.com/2012/06/a-closer-look-recent-email-based-malware-attacks/

I would say the most useful aspect of AV software is for scanning and cleaning attempts after an infection rather than pretending to offer 100% preventative security.

von JP - am 28.04.2013 11:49
If it is the stop.txt file is not found when starting the software, may be, there could be another reason.

If the UAC (use account control) is used on the pc and the software writes the stop.txt file in a 'forbidden' directory, it is written in the virtual store of windows. A sub-sub-.. directory of the \users\myuser\ directory. But when the software is reading the file, it is read at the place where the file originally was ment to be written. So the file is written, but at startup not found.

von Stefan Bentvelsen - am 29.04.2013 11:25
Hello Frans

I had a thought - always a dangerous thing.
In my software I have a process where users can make a copy of their data and send it back to me for problem solving. It runs from within the Windev application and is a simple Windev archive. The point is that from time to time one of the files in the archive is always corrupt. I am not sure if you would classify it as an index issue but if you open the file in WDMap and add records using Ctrl A it creates new records with identical primary keys (autoid fields) to the last one or two records and once you create those duplicate records the file is fine, without needing to be reindexed. WDMap has always been able to add duplicate records to a damaged file when you can't do it in the software - it must be able to suspend the integrity checking.

If you make the assumption that somehow the archive process is corrupting the open Hyperfile by writing to the archive while the Windev Hyperfile is open or in use then your process to write the Stop.txt file could in some way be causing your problem.
Do you use information from the file that is corrupt in your txt file?
Is it always the same file that is corrupt ?

A simple way to test this theory would be to not write the txt file but use a Windev file to hold the "stop" information as some others have suggested in this thread.

Regards
Al

von Al - am 01.05.2013 13:51
Hi ! I have found some info on internet regarding problems between Windows versions and the SMB protocol used. There is a way to "fix" this kind of situation, by downgrading ALL the PCs and servers to communicate using SMB1.

Here is the link with details on how to proceed:

http://www.petri.co.il/how-to-disable-smb-2-on-windows-vista-or-server-2008.htm

Have fun. Michel.

Message forwarded from pcsoft.us.windev

von Michel GARCIA.pcs.crosspost - am 02.05.2013 06:38
Frans,

The most important thing with ISAM database is to disable OPLOCKS under Windows !

Here is a good link for the explanation :
http://www.dataaccess.com/whitepapers/opportunlockingreadcaching.html

HTH
Sylvain

Message forwarded from pcsoft.us.windev

von Sylvain ROBERT ( Dat).pcs.crosspost - am 02.05.2013 06:42
Zur Information:
MySnip.de hat keinen Einfluss auf die Inhalte der Beiträge. Bitte kontaktieren Sie den Administrator des Forums bei Problemen oder Löschforderungen über die Kontaktseite.
Falls die Kontaktaufnahme mit dem Administrator des Forums fehlschlägt, kontaktieren Sie uns bitte über die in unserem Impressum angegebenen Daten.