Analysis

The Analysis tab lets you explore the data that has been imported from your log files into a Storage. The Analysis tab introduces a few important concepts such as Summaries and Schemas, and provides a great playground for summarizing and searching your log data, applying Aliases and identifying the information you'd like to extract using a Report Template.

What is a Summary?

A Summary can often be thought of as a field in your log file, such as Usernames. But Summaries can also utilize multiple fields in your log files to show a more intelligent result. For example, the Origin Domain summary uses the Fastvue Site Clean engine to process multiple fields in your log files and output a list of more 'sensible' web sites that users were actually visiting, rather than showing advertising, widgets, CDNs and other background sites.

Vantage can also take a single field in your log file to create multiple Summaries. For example, Vantage will split a single URL field into many Summaries such as Site Domain, Site Resource and Site Keywords (see Summaries created from URLs), or a DateTime field into multiple summaries such as Hours, Months and Years.

Summaries are the building blocks for your report templates and the objects that you can apply Aliases to.

What is a Schema?

A schema is the list of all the Summaries that Vantage Ultimate creates from the fields available in log file. Schemas are used to separate logically different sets of information from a log file.

For example, a log file from a UTM device may contain both Web and Email information. These two types of information are separated into different Schemas because it does not makes sense to drilldown from one into the other. For example, drilling down into an email subject line and into the web URLs associated with those email subjects does not make sense.

Depending on the log file format you are importing, the list of Summaries that are returned will vary. For example, when you import a Microsoft Exchange log file, you will see Summaries such as Senders, Recipients, and Subjects. When importing a Web Gateway log file, you will see Summaries such as Site Domain, Users and Action (Blocked/Allowed etc).

When you create Report Templates, you first need to select the Schema you want to report on, and then you can utilize all the Summaries in that schema to build out the Report content.

Using the Analysis tab

The Analysis tab enables you to analyze these Summaries and drilldown into your imported data to find specific information.

Running an Analysis

To begin analyzing your data, you need to run an Analysis. This launches the Analysis Wizard, which will guide you through the process of selecting the Storages, Schemas and Summaries to analyze.

Once your analysis is complete, the list of Summaries are shown, and clicking a Summary (such as Usernames) displays its values (such as domain\john.smith, domain\jane.doe etc).

To run an Analysis:

  1. Go to the Analysis tab
  2. Click New Analysis
  3. Select the Storage and Schema that you want to Analyze. Click Next.

  4. There are two types of Analyses:

    • Ad-hoc Analysis:
      An Ad-hoc Analysis displays all Summaries in your selected Schema and is a great way to start exploring your data. It is very handy having an ad-hoc analysis available on the Analysis tab while creating or customizing your templates, and/or creating Aliases.

      Precalculated Analysis
      When running an ad-hoc analysis, you will notice and option to 'Use Precalculated Analysis if Available'. To use this feature, you need to enable Analyze on import. See Precalculated Analysis for more information.

    • Template Analysis:
      If you know what you want to analyze, such as the top 10 users and the largest file downloads, you can create a Report Template that displays just this information, and select it when running an analysis. This is effectively the same as running a report, only the results are displayed on the Analysis tab, allowing you to use the drilldown functionality to drilldown beyond what has been defined in your report template.

  5. The Filters page enables you to filter your analysis by dates, times and field values in your log files.

    It is recommended that you analyze relatively small amounts of data, such as a day or a week, as the time it takes to perform each drilldown is affected by the amount of data Vantage Ultimate needs to analyze. To do this, click Add | Date Filter and select the dates you need to analyze. For more information see Filtering.

  6. The Summaries page lets you select the Summaries you want to see in your analysis. Typically, you can just leave the default settings and/or select all Summaries. You can also click the Add Summary button to create your own Summaries that are not defined in your Schema using the WebSpy Vantage Expression Language.

  7. Click OK to run the analysis.

Once your analysis has been run, you can interactively explore and drilldown into your data.

Drilldowns

You can drilldown further into your data by right-clicking any hyperlinked value and selecting Drilldown from the pop-up menu. You can now explore all the Summaries filtered by the value you have drilled down into. For example, drill down into a Username to the Origin Domain Summary to see the websites that user visited. Then click the Hour Summary at the same drilldown level to see what hours they do most of their browsing.

Aliases

Any Aliases that can be applied to the current summary can be selected from the Aliases task pad on the left. This transforms the values (such as domain\john.smith) into the aliased values (such as John Smith). You can add any item to an alias or by right-clicking the item and Add to alias.

To automatically create User and Department aliases based on information already stored in Active Directory, use the Import Organization from LDAP wizard, available on the Organization tab, or as a Task action.

Profiles

Vantage comes with a feature called Profiles for categorizing web URLs based on keywords. For example, a URL containing the word 'poker' will be profiled as Gambling. The results of URL keyword profiling are returned in the Site Profile Summary. You can configure Profiles and Keywords on the Profiles tab in Vantage, and you can also add keywords to profiles as you come across them on the Analysis tab.

Simply right-click a Summary value (such as the Origin Domain value 'facebook.com') and select Include in profile (such as 'Social Media') or Exclude from profile (such as 'News and Reference'). You also get the option to edit the keyword, enabling you to add just 'facebook' instead of 'facebook.com' as a keyword for example.

Extensions

You can also use an Extensions to send Summary values such as an IP address to an external command such as nslookup or ping by right-clicking any value and selecting the Extensions option. You can manage your extensions in Tools | Options | Analysis.

The available default extensions include:

ping Ping sends ICMP echo request packets to the target host and listens for ICMP echo response replies. The output includes statistics for approximate round trip times and includes information such as number of packets sent, received and lost.,

nslookup Nslookup (Name Server Lookup) is used to find details relating to DNS. The output includes the name and IP address of the machine being looked up. Note, you can also use Vantage's Resolve IPs functionality to lookup IPs in bulk and store the results in an Alias.

traceroute Traceroute sends out ICMP echo packets with increasing TTL (time-to-live) values to discover the path taken by IP Packets..

whois Whois is a query and response protocol that is widely used for finding who is the owner or the entity responsible for a domain name or an IP address.

After you have run an analysis you can right-click any value and select the Extensions menu to pass the value to the desired/

You can also add your own custom extensions in Tools | Options | Summaries.

Exporting Data

You can export the values from the selected Summary to a web document (HTML), word document (DOC), spreadsheet (CSV), PDF or text (TXT) formats, by clicking the Export current view link in the Summaries task pad.

You can export all Summaries at any given level by selecting their containing folder in the Summary Tree and clicking the Export all views link in the Summaries task pad.