OutWit Hub for Firefox - Version History
- Version 3.0.3.64
- Fix - Addressed performance issues in tutorial animations due to changes in Firefox 20.
- Fix - Corrected specific problem in scraper functions (#CONCAT#...) in case of values starting with ">".
- Fix - Corrected an issue with command line parameters.
- Version 3.0.3.60
- Feature - Three new items were added to the datasheets right-click menu: "Use row as headers", "Delete Block" and "Delete Other Blocks".
- Fix - Problems with column order were occurring in very specific cases when data was sent to the Catch. They were fixed.
- Fix - An HTML encoding problem on some characters was fixed in the POST query function.
- Enhancement - Small enhancements were brought to the 'Clean Text' function.
- Enhancement - Save macro dialogs are not shown anymore when application is run from the command line.
- Version 3.0.3.56
- Fix - Several interface glitches were corrected in datasheets.
- Enhancement - More cases handled by the autopilot for rare server errors that could occur during a browse or a dig, to prevent automatic explorations from stalling.
- Version 3.0.3.52 - Compatibility Update - Necessary for Firefox 19+
- Fix - Catch - A change in Firefox 19+ caused the catch to not be preserved between sessions. This was fixed.
- Fix - Scrapers - #BACK# and #FORWARD# are behaving like they used to in versions prior to 3.0.3.x.
- Fix - Scrapers - In 'keep order' mode, columns that have been split into several columns are now properly removed.
- Enhancement - More server errors are handled during automatic explorations, reducing the risk of broad Digs stopping in case of unexpected http errors.
- Enhancement - Works with Firefox 4 to 20.
- Version 3.0.3.28
- Known Issue - Scrapers - A regression in 3.0.3.28 prevents the URL variables #BACK# and #FORWARD# from working properly. This problem was fixed as soon as it came to our attention and the next build will include the correction.
- Feature - Guess - In the workshop, when list is checked, guess now displays the data as a record per row, a field per column.
- Fix - Datasheet - A problem was fixed that prevented in certain cases to add several columns with identical headers in a datasheet.
- Fix - Datasheets - Fixes in First/Last names split function.
- Fix - Scrapers - A problem was corrected in the instantiation of #URL# in replacement functions like #getParam()#, #setParam()#.
- Enhancement - Scrapers - #MIN#, #MAX#, #AVERAGE#... now recognize and use figure separators in calculations (thousands and decimal).
- Enhancement - New preferences were added: to allow the program to force the extractions on AJAX events and to force 'Ordinal ID' and 'Source URL' columns to be visible and present in the exports.
- Version 3.0.2.23
- Feature - A preference now allows you to disable the system that prevents automatic explorations from visiting the same URL several times. It may prove useful in some very particular cases with javascript links. In general you should not uncheck this preference.
- Feature - Two directives were added to override the Clean Text checkbox in the scraped view: #cleanData# and #originalData#.
- Fix - The status bar evaluation of remaining time in automatic explorations was sometimes exceeding 200,000 days. Such pessimistic calculations should not be trusted. The software is provided without warranty (express, implied or statutory) but, if there was a warranty, it couldn't exceed a century, two at most.
- Version 3.0.2.21
- Fix - Fixed a Download Manager problem which appeared with Firefox 18.
- Fix - Row deduplication functions are now ignoring the ordinal ID (always) and Source URL (preference). It should be more useable in the data extractors.
- Fix - Using directives in 'keep order' mode doesn't generate extra columns anymore.
- Version 3.0.2.19
- Fix - New fixes and enhancements in jobs.
- Fix - Fixed possible problems that could occur in some cases because of the recent addition of underscores ("_") among the accepted characters in field names.
- Fix - Replaced spaces (" ") by underscores ("_") in default field names in the scraper editor.
- Version 3.0.2.15
- Fix - Several fixes and enhancements in the job execution engine, preferences and management.
- Fix - Splitting cells to columns could fail in the case of untitled columns. It should now be working in all cases.
- Fix - #addToQueue# directive and #nextToVisit()# function now also works with javascript links.
- Enhancement - Checking/unchecking OKs on a multiple selection is now possible in the query editor like in the managers.
- Version 3.0.2.5
- Feature - Scrapers: added #FIRST#, #LAST#, #UNIQUE#, #STRICTLY-UNIQUE#, #DISTINCT#, #STRICTLY-DISTINCT#, #HAPAX#, #SHORTEST#, #LONGEST# replacement functions to the scraper engine.
- Fix - Scrapers: corrected a problem that could cause a browse to stop before the queue of URLs to visit had been completely explored. This could happen when a #nextPage# directive was setting a next page link that had already been visited.
- Feature - More User Gear management features were added to the File menu.
- Feature - Regular expressions are now allowed in the datasheet Right-Click Edit>Replace... functions.
- Fix - First name recognition was reactivated (colorizing, separation First/Last names are operational).
- Fix - Enhancements and fixes in the export layout management.
- Fix - The links extractor now properly handles 'Area' node links (image maps).
- Enhancement - If data was grabbed from several pages of the same site, the default HTML export title only displays the domain name instead of the full URL.
- Enhancement - Works with Firefox 4 to 18.
- Enhancement - The underscore character (_) is now allowed in datasheet column headers and scraper field names.
- Enhancement - Added a direct access to the FAQ from the Help menu.
- Version 3.0.1.47
- Fix - Corrected a problem causing partial document extraction.
- Version 3.0.1.45
- Feature - It is now possible to import the automators to the current User Gear from a backup User Gear file.
- Fix - Corrected the slow saving of jobs that was happening in some cases.
- Version 3.0.1.40
- Fix - Drag and drop is working again in the query editor (right panel).
- Fix - Column order and numbering is now working in 3.0 in the same way as in 2.1.
- Enhancement - Added Preferences to add DROP and CREATE instructions before the INSERTs in SQL Export.
- Enhancement - It is now possible to run a Job immediately, by choosing 0 minutes in the 'start in' menu.
- Enhancement - By default, the program asks to execute missed jobs at startup. A preference was added to prevent this.
- Version 3.0.1.34
- Fix - The option checkboxes were inactive in views bottom panels. This was corrected.
- Enhancement - The combined Browse & Dig process was fixed and optimized
- Feature - It is now possible to put catch/columnName in the start page textbox of the Macro editor.
- Version 3.0.1.33 - Release (Necessary update for Firefox 17+)
- Enhancement - Compatibility: Version 17 of Firefox dropped E4X, an important technology used to handle XML data. Several parts of the code were refactored to replace this library. This should not generate any changes in the behavior. Please notify us of any regression you may observe in the reading or saving of automators or in the execution of tutorials.
- Enhancement - Automators: The User Gear (the file containing your scrapers, macros etc.) is now backed up at launch each time its size has changed.
- Enhancement - Automators: Trying to import automators with an & in the path was failing. This was fixed.
- Feature - Automators: The User Gear containing your automators, export layouts and thesaurus can now be saved/backed up on the hard disk and loaded into the application. This allows you to easily transfer your scrapers, macros... from one profile to the other or from the addon to the standalone version.
- Fix - Automators: Fixed activation/deactivation of automators when clicking on an active checkbox in a multiple selection.
- Fix - Command Line: Macros based on a local URL now also work when launched in command line.
- Features - Datasheets: The right-click menu of all datasheets in the application was reorganized and now includes a number of additional features.
- Feature - Datasheets: A new "Delete Unselected" item was added to the datasheets right-click menu. (Instead of doing Invert Selection then Delete.)
- Feature - Datasheets: A new "Select Block" item was added to the datasheets right-click menu. It will select all the rows of the table/list/rss feed (depending on the view) where the selected row is located.
- Feature - Datasheets: A series of features (colorization button, people name recognition & split, etc) were added to the datasheets. They are destined to be used with a simple user thesaurus feature which will be added in a later version. We already included them as they may prove interesting separately.
- Feature - Datasheets: New options of the datasheet right-click menu allow you to delete colums with few populated cells. (This is interesting after large scrapes, when useless columns have be created by marginal data fields.)
- Features - Datasheets: Replacement functions were added to the right-click menu on datasheets.
- Enhancement - Datasheets: Multiple optimizations were brought to datasheets behavior and coloring when selecting / modifying cells or rows.
- Enhancement - Datasheets: The Normalize Figures function was enhanced and can now be applied on a whole datasheet.
- Enhancement - Datasheets: The number of columns allowed in a row of any datasheet is now limited to 120. This value is arbitrary you can change it the preference panel. Note, however, that performances decrease dramatically with several hundreds of columns.
- Fix - Datasheets: Column headers are now cleared when the data is deleted from a datasheet.
- Enhancement - Detail: The Detail panel was modified to be able to display even very large fields in very large records.
- Enhancement - Dig: Added sql, csv to the list of diggable file types.
- Feature - Export: An Export Preview panel is available in all views on the left of the datasheet.
- Enhancement - Export: The SQL export was corrected and enhanced. It is now possible to define an export file name that includes a . (dot), allowing you to address a database as well as the table: if you name the file myDB.myTable.sql, the insert statements will be done to the database myDB and the table myTable.
- Enhancement - Export: Added thead/tbody to table export.
- Feature - Export: The Edit button at the top right of the Export Preview panel gives access to the Export Layout Editor, where you can choose which column should be exported, in what order, with what separator and in what style.
- Fix - Exports: Preferences now have the priority over the default export layout for separators in CSV export.
- Enhancement - Fast Scraping: Added a cache in fast scraping, which means that the application will not send twice the same query during a fast scrape.
- Enhancement - Fast Scraping: The Fast Scrape function was dramatically optimized for large numbers of queries. This should not be noticeable for casual use, but if you scrape tens of thousands of URLs or more in one batch, it will.
- Enhancement - Guess: Import / recognition of separated or tabulated text files (CSV, TXT, SQL...) by the guess view was enhanced.
- Enhancement - Guess: Improved date & people name recognition.
- Enhancement - Guess: The guess view is understanding more data files formats and should handle better field contents which include separators.
- Fix - Help: Fixed navigation buttons in the Help Panel.
- Enhancement - Install: It is now possible to install or update the Hub directly by clicking on the link of an .xpi file.
- Enhancement - Install: The Mac version is now digitally signed (like the Windows version) to avoid alerts in Mountain Lion.
- Enhancement - Install: Works again on 32bit Macs as well as 64bits.
- Enhancement - Jobs: Jobs with a past date are now automatically unchecked.
- Feature - Jobs: Jobs can now contain multiple macros. Several actions can also be included in a job (reload, reapply, quit, pause...).
- Fix - Jobs: Fixes in job execution after quiting and restarting the application.
- Feature - Lists: <title> and <h1>, <h2>... tags can now be included in the lists extraction result, when the "Add Titles" option is checked in the "lists" bottom panel.
- Enhancement - Log: It is now possible to copy data from the log panel.
- Enhancement - Log: The Log Panel at the top of each view can slide towards the right and give room to the current page so that you see in the same screen the original page and the data being extracted.
- Enhancement - Macros: Cosmetic work on the Macro editor (serious refactoring is still overdue, but this is already better).
- Fix - Macros: Activation/deactivation of scrapers in the execution of macros is behaving slightly differently from 2.1. Please look for side effects in your previous macros.
- Fix - Macros: Editable menu lists are back in the macro editor to select dynamic columns in data extractors.
- Fix - Macros: In rare cases, macros could stall randomly during execution, due to redirections in social network iFrames (facebook likes, etc) A monitoring module was added which reactivates the macro in these cases.
- Features - Ordinal: A new 'Ordinal' column was added to all datasheets (hidden by default, check it in the column picker in the upper right corner to see it). This column contains an id composed of three numbers separated by dots. The first number is incremented each time a new extraction is done when the 'empty' checkbox is not checked. The second number is only used in lists, tables, scraped and news: it is incremented for each new list, table or rss feed in the document, the third number is the order number of the occurrence of the item in the document. This can be very useful to select specific items or ranges of items with the 'selectIf' textbox.
- Enhancement - Ordinal: The ordinal ID of fast scrape results has changed to make it match the format of the other views. What was the ordinal ID in previous versions is now the second group of digits in the new format.
- Feature - Page: The 'Workshop' option of the Navigation menu gives access to a blank page where you can paste data to be processed by the Hub. When you right-click in the page, note the 'Paste Text' option of the menu: it allows you to paste data from Excel, Word or other applications without formating. (Guess should do a better job with data pasted this way from spreadsheets.)
- Enhancement - Page: Added sql to the list of accepted file extensions. SQL files can now be opened and processed with OutWit Hub.
- Enhancement - Performances: Link extraction now takes almost half the time on page load. This is noticeable on very large pages like the United States article on Wikipedia.
- Enhancement - Performances: Improved loading time when the Catch contains rows with large numbers of column.
- Enhancement - Performances: The addition of large blocks of data to a sorted datasheet has also been made dramatically faster.
- Enhancement - Preferences: A download rename pattern (preference) was added: filename truncation option [original name,10] (first 10 chars) or [original name, -30] (last 30 chars).
- Enhancement - Preferences: The default setting (ignoring non-latin characters, in the words view) was modified to work better with Central European languages.
- Feature - Preferences: A new preference was added to the Advanced panel, allowing you to allow the 'words' views to extract words in non-latin characters. (This will be refined as we go, following our users feedback.)
- Feature - Preferences: Added preference for replacement of return characters in 'Clean Text'.
- Feature - Preferences: Added preference to ignore HTTP errors while Fast Scraping.
- Fix - Preferences: Corrected temporization scale imprecisions in time preference.
- Fix - Preferences: The 'Bypass Cache' preference is working again (it could not be changed and remained set to 'always').
- Feature - Proxy: Proxy setting can now be defined within the Hub.
- Fix - Registration: With profiles containing hundreds of automators, the program could start in light mode and switch to pro after a few seconds. This was corrected.
- Feature - Scrapers: New replacement functions were added to scrapers: #MAX#, #MIN#, #AVERAGE#, #SUM#, #CONCAT# and #COUNT#. (see tutorial.) These should be entered in the replacement field of a scraper line to get one cumulated/computed value instead of all the matching values.
- Enhancement - Scrapers: It is now possible to set the value of a scraper variable with a variable or function.
- Enhancement - Scrapers: Several enhancements and fixes were brought to the scraping engine (please test your old scrapers).
- Enhancement - Scrapers: The Scraper code has changed (please test your old scrapers), mostly in the way cell text is cleaned. Performances should be better in many cases.
- Enhancement - Scrapers: When a scraper has a higher priority than the one you are trying to use, the dialog now allows you to disable it and run the one you chose.
- Feature - Scrapers: added #check# directive (same as #checkIf# and #checkIfNot#, but without a condition).
- Feature - Scrapers: added #keepOrder# directive which allows you to set this option in the scraper rather then manually or in a macro.
- Feature - Scrapers: added #setAnchorRow# #useAnchorRow# #useCurrentRow# directives which allow you to scrape columns in HTML pages that are made using left or right floating blocks.
- Feature - Scrapers: New directives were added to scrapers. #caseSensitive#, #checkIf#, #checkIfNot#, #setDLRenamePattern#, #setExportRenamePattern#, #insertIf#, #insertIfNot#, #setPauseInterval#, #setPauseDuration#...
- Features - Scrapers: A #pause# directive was added to the scrapers (enter the pause duration in seconds in the replacement field).
- Features - Scrapers: The directive #addToQueue# was added to the scrapers. It allows to grab URLs to be visited without using a variable, and to use the #nextToVisit()# without a parameter.
- Features - Scrapers: The directive #showNextPageCandidates# was added to the scrapers, for debug purposes.
- Fix - Scrapers: #TIME#, #DATE#, #DATETIME# are now working even if 'before', 'after' and 'format' fields are empty in a scraper line.
- Fix - Scrapers: Activating / deactivating scrapers was not working properly in the light version.
- Fix - Scrapers: Corrected problem happening in some cases when using \0 in a #replace# scraper directive.
- Fix - Scrapers: Fixed a recent bug with #URL# and #BASEURL# in scrapers.
- Enhancement - Source: Source code extraction and colorization was optimized. The source display time should be divided by three or more compared to v2.1.
- Enhancement - Source: Source colorization has changed slightly. Image links are now appearing in dark yellow.
- Feature - String Generation Patterns: A more efficient editor was added for creating string generation patterns (matrices).
- Feature - String Generation Patterns: In enumerations like [a;b;c][1;2;3;4] added modulo fill when group patterns do not have the same length (instead of showing error).
- Feature - String Generation Patterns: It is now posible to generate alternated strings for cases where you may need to go back to a menu between pages, for instance.
- Feature - String Generation: Added hexadecimal and alphanumerical string generation (like in [00:FF]).
- Enhancement - Tables: In the table extractor, the algorithm deciding which link to put in the first column was modified to select the most interesting link (in most cases).
- Enhancement - Tutorials: Since Firefox 11, on Macintosh, a bug was preventing UI element animations in tutorials and wizards. To highlight interface items, we have now replaced the lens effect on Mac by the circling of designated elements and reactivated the feature.
- Fix - Tutorials: Fixed highlighting and animations in tutorials.
- Enhancement - UI: A time estimate was added in the status bar during automatic explorations.
- Enhancement - UI: The program interface was somewhat altered, in particular to allow for Export Layout personalization.
- Fix - More fixing, cleaning, debugging, optimizing, refactoring, beautifying and all that sort of things.
