If you use Angelfish Software v2.4+, please use the new Help Center:
https://help.angelfishstats.com/helpdesk/?aa=200
DATES IN WEB LOGS
Web log filenames usually contain the date the file was created. For example, IIS servers create a daily log and use the date in the filename, like this:
u_ex140110.log
This log was created on January 10, 2014. The "140110" in the filename represents the 2 digit year, month, and day of the file creation date. Moving forward, the filename from January 11 will contain "141011", the filename from January 12 will contain "140112", and so on.
Therein lies the problem. If every new log file created has a different name, how do you configure Angelfish to automatically "see" each new file?
DATE SUBSTITUTION
Date substitution statements automatically adapt to an ever-changing date, like this:
u_exYYMMDD.log
"YY" represents the current 2 digit year
"MM" represents the current 2 digit month
"DD" represents the current 2 digit day
During processing, Angelfish will replace any date substitution statements with the current year/month/day and look for files that match the pattern. If the current date is Jan. 10, 2014, the above example will match any file named "ex140110.log"
Date Substitution Offset
Datasources contain an offset setting which adjusts date substitution by a number of days. By default, the setting is "-1" which will substitute yesterday's date (i.e. the current day, minus one). If the current date is Jan. 10, 2014 and the offset is -1, "u_exYYMMDD.log" will match any file named "ex140109.log"
DATE SUBSTITUTION SYNTAX
Commonly-used date substitution syntax is below:
- YYYY The current 4 digit year
- YY The current 2 digit year
- MM The current 2 digit month
- DD The current 2 digit day
- HH The current 2 digit hour
- %d The day of the month as a decimal number (range 01 to 31).
- %H The hour as a decimal number using a 24-hour clock (range 00 to 23).
- %I The hour as a decimal number using a 12-hour clock (range 01 to 12).
- %m The month as a decimal number (range 01 to 12).
- %y The year as a decimal number without a century (range 00 to 99).
- %Y The year as a decimal number including the century.
Angelfish uses the strftime() function, which accepts additional abbreviations.
WILDCARDS
The regular expression .* (a.k.a. "dot star") roughly translates to "match everything". If you want to match all files in a directory, you can use /.* as your match pattern in the location field.
But what if you only want to match a certain list of files, like files from the year 2014? Simply append the wildcard to the match pattern, like this:
u_ex14.*
You can also combine wildcards and date substitution statements in the same field, like this:
u_exYYMM.*
BEST PRACTICES
Ddate substitution statements are most useful when combined with a wildcard, because they automatically compensate for errors. For example...if you use u_exYYMMDD.log for date substitution and your Angelfish server takes a nap over the weekend, you'll have to update each Datasource on Monday so the skipped days are processed.
A better approach is to use u_exYYMM.* - this statement will match any file from the current year and month. If your Angelfish server crashes on the 12th of the month and isn't revived until the 15th, Angelfish will automatically process the missing logs!
In addition to wildcards, there are other Regular Expressions you can use with date substitution.
EXAMPLES
Consider the following list of log files, which shows the first of the month from Jan 2013 - December 2013
u_ex130101.log
u_ex130201.log
u_ex130301.log
u_ex130401.log
u_ex130501.log
u_ex130601.log
u_ex130701.log
u_ex130801.log
u_ex130901.log
u_ex131001.log
u_ex131101.log
u_ex131201.log
If the current date is December 10, 2013:
- u_exYY.* matches all files from 2013 (all files listed)
- u_exYYMM.* matches all files from December 2013 (only u_ex131201.log)
- u_exYYMMDD.* matches all files from December 9, 2013 (no file exists)
0 Comments