sk8rmatt59p
New Member
I have been asked to write a document management system for the organisation where I work which offers a series of nine different workflows related to different records. Amongst the workflows is adding documents to the "file" or record, and publishing a sub-set of these documents to the public website based on business rules.The documents are almost without exception in PDF format, and usually fewer than twenty are being processed for any one record, and at any one time.A primary reason for building this as a web application was to keep the files in our data center and on high speed switches, rather than try and copy up and back down between locations via potentially slow connection speeds at remote sites.The system has worked flawlessly until a larger series of documents (114 PDF documents totally 329MB in size) timed out around 95% of the way in.The code is (IncomingDocuments is of type List<FileInfo>)-\[code\]List<string> filesSuccessfullyAdded = new List<string>();foreach (FileInfo incomingFile in IncomingDocuments){ FileOperations.AddDocument(incomingFile, false, ApplicationCode, (targetDirectoryPath.EndsWith(@"\") ? targetDirectoryPath : targetDirectoryPath + @"\")); FileInfo copiedDocument = new FileInfo(Path.Combine(targetDirectoryPath, incomingFile.Name)); if (copiedDocument.Exists && copiedDocument.Length == incomingFile.Length && copiedDocument.LastWriteTime == incomingFile.LastWriteTime) { filesSuccessfullyAdded.Add(copiedDocument.Name); }}if (filesSuccessfullyAdded.Any()){ SetupConfirmationLiteral.Text += "<p class='info'>The following files have been successfully added to the application file-</p>"; XDocument successfullyAddedList = new XDocument( new XElement("ul", new XAttribute("class", "documentList"))); foreach (string successfulFile in filesSuccessfullyAdded) { successfullyAddedList.Root.Add(new XElement("li", successfulFile)); } SetupConfirmationLiteral.Text += successfullyAddedList.ToString();}var notSuccessfullyAdded = from FileInfo incomingDocument in IncomingDocuments where !filesSuccessfullyAdded.Contains(incomingDocument.Name) orderby incomingDocument.Name ascending select incomingDocument.Name;if (notSuccessfullyAdded.Any()){ SetupConfirmationLiteral.Text += "<p class='alert'>The following files have <strong>not</strong> been successfully added to the application file-</p>"; XDocument notAddedList = new XDocument( new XElement("ul", new XAttribute("class", "documentList"))); foreach (string notAdded in notSuccessfullyAdded) { notAddedList.Root.Add(new XElement("li", notAdded)); } SetupConfirmationLiteral.Text += notAddedList.ToString(); SetupConfirmationLiteral.Text += "<p>A file of the same name may already exist in the target location.</p>";}\[/code\]With a utility method of-\[code\]public static void AddDocument(FileInfo sourceFile, bool appendName, string applicationCode, string targetPath){ try { DirectoryInfo targetDirectory = new DirectoryInfo(targetPath); if (targetDirectory.Exists) { string targetFileName = (appendName ? sourceFile.Name.Insert(sourceFile.Name.IndexOf(sourceFile.Extension, StringComparison.Ordinal), " UPDATED") : sourceFile.Name); if (targetDirectory.GetFiles(targetFileName).Any()) { //Do not throw an exception if the file already exists. Silently return. If the file exists and matches both last modified and size it won't be reported, and can be archived as normal, //otherwise it should be reported to user in the calling method. return; } string targetFileUnc = Path.Combine(targetPath, targetFileName); sourceFile.CopyTo(targetFileUnc, overwrite: false); Logging.FileLogEntry(username: (HttpContext.Current.User.Identity.IsAuthenticated ? HttpContext.Current.User.Identity.Name : "Unknown User"), eventType: LogEventType.AddedDocument, applicationCode: applicationCode, document: sourceFile.Name, uncPath: targetFileUnc); } else { throw new PdmsException("Target directory does not exist"); } } catch (UnauthorizedAccessException ex) { throw new PdmsException("Access was denied to the target directory. Contact the Service Desk.", ex); } catch (PathTooLongException) { throw new PdmsException(string.Format("Cannot add document {0} to the Site File directory for Application {1} - the combined path is too long. Use the Add Documents workflow to re-add documents to this Site File after renaming {0} to a shorter name.", sourceFile.Name, applicationCode )); } catch (FileNotFoundException ex) { throw new PdmsException("The incoming file was not found. It may have already been added to the application file.", ex); } catch (DirectoryNotFoundException ex) { throw new PdmsException("The source or the target directory were not found. The document(s) may have already been added to the application file.", ex); } catch (IOException ex) { throw new PdmsException("Error adding files - file(s) may be locked or there may be server or network problem preventing the copy. Contact the Service Desk.", ex); }}\[/code\]To do the actual copy and audit. The PdmsException is just a specific exception class we use to show helpful error messages to the user, allowing them to solve their own problem where possible, or at least give an understandable reason for the failure.I know I can simply increase the ExecutionTimeout property (http://msdn.microsoft.com/en-us/lib...tion.httpruntimesection.executiontimeout.aspx) beyond the default of 110 seconds - up to say 300 seconds - and this will probably mean that timeouts stop happening in this case, but what if a user is trying to add or publish thousands of documents. This solution will not scale well, simply deferring rather than solving the problem.I am using .NET 4 with Visual Studio 2010 and as far as I know will therefore have to use third party implementations of async and await like AsyncBridge (https://nuget.org/packages/AsyncBridge) if we want to parcel up the documents and update progress using ajax. I do not have access to Visual Studio 2012 or even Windows more recent than XP in order to use the Microsoft provided Async targeting pack.Given these constraints, is there a way to parcel up/batch process these documents to avoid timeouts and (ideally) provide feedback to the user as each batch is added? I am open to exploring F# if it would be straightforward to implement. Alternatively should I be pleading the case for Visual Studio 2012 on this one?