Skip to main content
RTILA Marketplace

On-Page SEO Auditor - output schema

Audit any URL list for indexability, metadata, readability, links, structured data and accessibility - on your own machine.

SEO Intermediate Stable v1.0.0 Updated Updated 2026-09-20 10 min read
Download
  • Windows x86_64

    BOT_on-page-seo-auditor.exe

    98.9 MB

    Download
  • macOS Apple Silicon chmod +x

    BOT_on-page-seo-auditor.sh

    87.5 MB

    Download
  • macOS Intel chmod +x

    BOT_on-page-seo-auditor.sh

    99.4 MB

    Download
  • Linux x86_64 chmod +x

    BOT_on-page-seo-auditor.sh

    122.5 MB

    Download

Do not rename the downloaded file - the filename carries the bot's configuration id.

This tab documents what a finished run produces: every dataset with its fields and transformations, sample rows, the files written to disk, and the destinations results are delivered to. Back to the overview tab.

The tables on this page are generated automatically from the published project configuration and therefore cannot disagree with it.

Datasets

One table per dataset: item selector, deduplication key, required fields, and per-field transformations.

Response_Indexability

Item selector: css=html - Required: Page_URL

Fields of dataset "Response_Indexability"
FieldProperty typeSelector / notesDefault
Page_URL page_url
Final_URL variable=last_final_url text
Status_Code variable=last_http_status text
Status_Text variable=last_http_status_text text
Navigation_OK variable=last_navigation_ok text
Navigation_Error variable=last_navigation_error text
Response_Time_ms variable=last_response_time_ms text
HTTP_Protocol variable=last_protocol text
Server_Header variable=last_server text
Content_Type variable=last_content_type text
Charset_Header variable=last_charset text
Content_Length variable=last_content_length text
Cache_Control variable=last_cache_control text
ETag variable=last_etag text
Last_Modified variable=last_last_modified text
X_Robots_Tag variable=last_x_robots_tag text
Redirect_Count variable=last_redirect_count text
Set_Cookie_Names text
Field details

Selector: variable=last_set_cookie_names

Transformations:

  1. script (code: if(!value)return '';if(Array.isArray(value))return value.join(', ');if(typeof value==='object')return Object.keys(value).join(', ');return String(value);, default_value: )

Redirect_Chain text
Field details

Selector: variable=last_redirect_chain

Transformations:

  1. script (code: if(!Array.isArray(value)||!value.length)return '';return value.map(function(h){return (h&&h.status?h.status:'?')+' '+(h&&h.url?h.url:'')+(h&&h.location?' -> '+h.location:'');}).join(' | ');, default_value: )

All_Response_Headers text
Field details

Selector: variable=last_response_headers

Transformations:

  1. script (code: if(!value)return '';try{return JSON.stringify(value);}catch(e){return String(value);}, default_value: )

Status_Class text
Field details

Selector: variable=last_http_status

Transformations:

  1. script (code: var n=Number(value);if(!isFinite(n)||n===0)return '0xx No Response';if(n<200)return '1xx Informational';if(n<300)return '2xx Success';if(n<400)return '3xx Redirect';if(n<500)return '4xx Client Error';return '5xx Server Error';, default_value: 0xx No Response)

Is_Success text
Field details

Selector: variable=last_http_status

Transformations:

  1. script (code: var n=Number(value);return String(isFinite(n)&&n>=200&&n<300);, default_value: false)

Is_Redirect text
Field details

Selector: variable=last_http_status

Transformations:

  1. script (code: var n=Number(value);return String(isFinite(n)&&n>=300&&n<400);, default_value: false)

Is_Client_Error text
Field details

Selector: variable=last_http_status

Transformations:

  1. script (code: var n=Number(value);return String(isFinite(n)&&n>=400&&n<500);, default_value: false)

Is_Server_Error text
Field details

Selector: variable=last_http_status

Transformations:

  1. script (code: var n=Number(value);return String(isFinite(n)&&n>=500);, default_value: false)

URL_Length page_url
Field details

Transformations:

  1. script (code: return String(String(value).length);, default_value: 0)

URL_Depth page_url
Field details

Transformations:

  1. script (code: try{return String(new URL(String(value)).pathname.split('/').filter(function(s){return s.length>0;}).length);}catch(e){return '0';}, default_value: 0)

URL_Protocol page_url
Field details

Transformations:

  1. script (code: try{return new URL(String(value)).protocol.replace(':','');}catch(e){return '';}, default_value: )

URL_Host page_url
Field details

Transformations:

  1. script (code: try{return new URL(String(value)).hostname;}catch(e){return '';}, default_value: )

URL_Path page_url
Field details

Transformations:

  1. script (code: try{return new URL(String(value)).pathname;}catch(e){return '';}, default_value: )

URL_Query_String page_url
Field details

Transformations:

  1. script (code: try{return new URL(String(value)).search.replace(/^\?/,'');}catch(e){return '';}, default_value: )

URL_Fragment page_url
Field details

Transformations:

  1. script (code: try{return new URL(String(value)).hash.replace(/^#/,'');}catch(e){return '';}, default_value: )

URL_Param_Count page_url
Field details

Transformations:

  1. script (code: try{return String(Array.from(new URL(String(value)).searchParams.keys()).length);}catch(e){return '0';}, default_value: 0)

URL_Param_Names page_url
Field details

Transformations:

  1. script (code: try{return Array.from(new URL(String(value)).searchParams.keys()).join(', ');}catch(e){return '';}, default_value: )

URL_Has_Parameters page_url
Field details

Transformations:

  1. script (code: try{return String(new URL(String(value)).search.length>1);}catch(e){return 'false';}, default_value: false)

URL_Has_Fragment page_url
Field details

Transformations:

  1. script (code: try{return String(new URL(String(value)).hash.length>1);}catch(e){return 'false';}, default_value: false)

URL_Has_Session_ID page_url
Field details

Transformations:

  1. script (code: return String(/(?:^|[?&;/])(phpsessid|jsessionid|aspsessionid[a-z]*|sid|sessionid|session_id|cfid|cftoken)(?:=|;|$)/i.test(String(value)));, default_value: false)

URL_Tracking_Params page_url
Field details

Transformations:

  1. script (code: var m=String(value).match(/[?&](utm_[a-z]+|gclid|fbclid|msclkid|dclid|gbraid|wbraid|twclid|igshid|mc_[a-z]+|yclid|_ga|_hs[a-z]+|vero_[a-z]+|ref|source|medium|campaign)(?==|&|$)/gi);if(!m)return '';var out=[];for(var i=0;i<m.length;i++){var k=m[i].replace(/^[?&]/,'');if(out.indexOf(k)===-1)out.push(k);}return out.join(', ');, default_value: )

URL_File_Extension page_url
Field details

Transformations:

  1. script (code: try{var p=new URL(String(value)).pathname;var m=p.match(/\.([a-z0-9]{1,5})$/i);return m?m[1].toLowerCase():'';}catch(e){return '';}, default_value: )

URL_Is_Non_HTML page_url
Field details

Transformations:

  1. script (code: return String(/\.(pdf|docx?|xlsx?|pptx?|zip|rar|gz|tar|7z|exe|dmg|csv|txt|rtf|epub|mp[34]|avi|mov|wmv|flv|webm|png|jpe?g|gif|svg|webp|ico|css|js|json|xml)(?:$|\?)/i.test(String(value)));, default_value: false)

URL_Word_Count page_url
Field details

Transformations:

  1. script (code: try{var p=new URL(String(value)).pathname;return String(p.split(/[\/_\-\.]+/).filter(function(s){return s.length>1;}).length);}catch(e){return '0';}, default_value: 0)

URL_Uses_Underscores page_url
Field details

Transformations:

  1. script (code: try{return String(new URL(String(value)).pathname.indexOf('_')!==-1);}catch(e){return 'false';}, default_value: false)

URL_Has_Uppercase page_url
Field details

Transformations:

  1. script (code: try{var p=new URL(String(value)).pathname;return String(p!==p.toLowerCase());}catch(e){return 'false';}, default_value: false)

URL_Is_Secure page_url
Field details

Transformations:

  1. script (code: try{return String(new URL(String(value)).protocol==='https:');}catch(e){return 'false';}, default_value: false)

URL_Is_Pretty page_url
Field details

Transformations:

  1. script (code: try{var u=new URL(String(value));return String(u.search.length<=1 && !/\.[a-z0-9]{2,5}$/i.test(u.pathname) && u.pathname===u.pathname.toLowerCase());}catch(e){return 'false';}, default_value: false)

Meta_Robots attribute
Field details

Selector: css=meta[name='robots']

Attribute: content

Meta_Robots_Count css=meta[name='robots'] count
Meta_Robots_All css=meta[name='robots'] text
Canonical_URL attribute
Field details

Selector: css=link[rel='canonical']

Attribute: href

Canonical_Count css=link[rel='canonical'] count
Meta_Refresh attribute
Field details

Selector: css=meta[http-equiv='refresh' i]

Attribute: content

Base_Href attribute
Field details

Selector: css=base[href]

Attribute: href

HTML_Lang attribute
Field details

Selector: css=html

Attribute: lang

Charset_Meta attribute
Field details

Selector: css=meta[charset]

Attribute: charset

Viewport_Content attribute
Field details

Selector: css=meta[name='viewport']

Attribute: content

Indexability list
Field details

Selector: css=self

Transformations:

  1. script (code: var it=(Array.isArray(value)&&value.length)?value[0]:{};function s(x){return (x===null||x===undefined)?"":String(x);}function lo(x){return s(x).toLowerCase();}function n(x){var v=parseInt(x,10);return isFinite(v)?v:0;}var pageUrl=s(it.page_url),finalUrl=s(it.final_url)||pageUrl;var statusNum=(it.status===null||it.status===undefined||it.status==="")?0:Number(it.status);if(!isFinite(statusNum))statusNum=0;var pu=null,fu=null;try{pu=new URL(pageUrl);}catch(e){}try{fu=new URL(finalUrl);}catch(e){}var metaRaw=s(it.metarobots),xRaw=s(it.xrobots);var combined=(metaRaw+", "+xRaw).toLowerCase();var has=function(d){return combined.indexOf(d)!==-1;};var directives=[];combined.split(",").forEach(function(d){d=d.trim();if(d)directives.push(d);});function dv(name){for(var i=0;i<directives.length;i++){var p=directives[i].split(":");if(p[0].trim()===name)return p.slice(1).join(":").trim();}return "";}var noindex=has("noindex")||has("none");var nofollow=has("nofollow")||has("none");var noindexSources=[];if(lo(metaRaw).indexOf("noindex")!==-1||lo(metaRaw).indexOf("none")!==-1)noindexSources.push("meta robots");if(lo(xRaw).indexOf("noindex")!==-1||lo(xRaw).indexOf("none")!==-1)noindexSources.push("X-Robots-Tag");var ct=lo(it.content_type);var isHtml=ct.indexOf("html")!==-1||ct==="";var isPdf=ct.indexOf("pdf")!==-1;var isDoc=!isHtml&&ct!=="";var blockers=[];if(statusNum===0)blockers.push("No HTTP response");else if(statusNum>=400)blockers.push("HTTP "+statusNum);if(noindexSources.length)blockers.push("noindex ("+noindexSources.join(" + ")+")");if(s(it.metarefresh).trim()!=="")blockers.push("meta refresh");if(!isHtml)blockers.push("Non-HTML content type: "+ct);var indexable=blockers.length===0;var canonRaw=s(it.canonical).trim(),canonCount=n(it.canonical_n);var canonAbs=/^https?:\/\//i.test(canonRaw),canonUrl="",cu=null;if(canonRaw){try{cu=new URL(canonRaw,pageUrl);canonUrl=cu.href;}catch(e){canonUrl=canonRaw;}}var canonSelf=(cu&&pu)?(cu.origin===pu.origin&&cu.pathname===pu.pathname&&cu.search===pu.search):false;var canonFinal=(cu&&fu)?(cu.origin===fu.origin&&cu.pathname===fu.pathname&&cu.search===fu.search):false;var canonIssues=[];if(!canonRaw)canonIssues.push("Missing");if(canonCount>1)canonIssues.push("Multiple ("+canonCount+")");if(canonRaw&&!canonAbs)canonIssues.push("Relative");if(canonRaw&&cu&&pu&&cu.protocol!==pu.protocol)canonIssues.push("Scheme mismatch");if(canonRaw&&cu&&pu&&cu.hostname!==pu.hostname)canonIssues.push("Cross-host");if(canonRaw&&cu&&cu.search.indexOf("=")!==-1)canonIssues.push("Has parameters");if(canonRaw&&!canonSelf&&!canonFinal)canonIssues.push("Points elsewhere");var mr=s(it.metarefresh).trim(),mrDelay=null,mrTarget="";if(mr){var m1=mr.match(/^\s*(\d+)\s*;\s*url\s*=\s*(.+)$/i);if(m1){mrDelay=parseInt(m1[1],10);mrTarget=m1[2].trim().replace(/^['"]|['"]$/g,"");}else{var m2=mr.match(/^\s*(\d+)/);if(m2)mrDelay=parseInt(m2[1],10);}}var hostChanged=(pu&&fu)?(pu.hostname!==fu.hostname):false;var protoUp=(pu&&fu)?(pu.protocol!==fu.protocol):false;var wwwChanged=(pu&&fu)?(pu.hostname.replace(/^www\./,"")!==fu.hostname.replace(/^www\./,"")):false;var slashChanged=(pu&&fu)?((pu.pathname.slice(-1)==="/")!==(fu.pathname.slice(-1)==="/")):false;var caseChanged=(pu&&fu)?(pu.pathname!==fu.pathname&&pu.pathname.toLowerCase()===fu.pathname.toLowerCase()):false;var pathChanged=(pu&&fu)?(pu.pathname!==fu.pathname):false;var paramsChanged=(pu&&fu)?(pu.search!==fu.search):false;var metaCharset=lo(it.charset_meta),hdrCharset=lo(it.charset_hdr);var charsetMatch=(!metaCharset||!hdrCharset)?null:(metaCharset===hdrCharset);var vp=s(it.viewport).trim();var vpPresent=vp!=="";var vpBlocks=vpPresent&&(vp.indexOf("user-scalable=no")!==-1||/maximum-scale\s*=\s*1(\.0)?\b/.test(vp));var cookies=Array.isArray(it.cookies)?it.cookies:[];var cookieStr=cookies.join(",").toLowerCase();return {indexable:String(indexable),indexability_verdict:indexable?"Indexable":"Non-Indexable",indexability_blockers:blockers.join("; "),noindex:String(noindex),noindex_sources:noindexSources.join(", "),nofollow:String(nofollow),noarchive:String(has("noarchive")),nosnippet:String(has("nosnippet")),noimageindex:String(has("noimageindex")),max_snippet:dv("max-snippet"),max_image_preview:dv("max-image-preview"),max_video_preview:dv("max-video-preview"),unavailable_after:dv("unavailable_after"),robots_directives_all:directives.join(", "),canonical_is_self:String(canonSelf),canonical_is_absolute:String(canonAbs),canonical_resolved:canonUrl,canonical_issues:canonIssues.join("; "),canonical_verdict:canonIssues.length===0?"OK":canonIssues.join("; "),host_changed:String(hostChanged),protocol_upgraded:String(protoUp),www_changed:String(wwwChanged),trailing_slash_changed:String(slashChanged),case_changed:String(caseChanged),path_changed:String(pathChanged),params_changed:String(paramsChanged),meta_refresh_present:String(mr!==""),meta_refresh_delay:mrDelay===null?"":String(mrDelay),meta_refresh_target:mrTarget,content_type_is_html:String(isHtml),is_pdf:String(isPdf),is_document_resource:String(isDoc),charset_matches_header:charsetMatch===null?"":String(charsetMatch),charset_declared:String(metaCharset!==""||hdrCharset!==""),viewport_present:String(vpPresent),viewport_blocks_scaling:String(vpBlocks),is_mobile_friendly:String(vpPresent&&!vpBlocks),html_lang_present:String(s(it.html_lang).trim()!==""),cookie_count:String(cookies.length),has_analytics_cookie:String(/_ga|ga_|_gid|_utm|_fbp|_fbc|_hj/.test(cookieStr)),has_session_cookie:String(/sess|jsessionid|phpsessid|sid|cfid|cftoken|asp\.net_sessionid/.test(cookieStr))};)

Nested list fields: page_url (page_url), final_url (text), status (text), xrobots (text), content_type (text), charset_hdr (text), protocol (text), metarobots (text), canonical (attribute), canonical_n (count), metarefresh (attribute), charset_meta (attribute), viewport (attribute), html_lang (attribute), cookies (text), indexable (index), indexability_verdict (index), indexability_blockers (index), noindex (index), noindex_sources (index), nofollow (index), noarchive (index), nosnippet (index), noimageindex (index), max_snippet (index), max_image_preview (index), max_video_preview (index), unavailable_after (index), robots_directives_all (index), canonical_is_self (index), canonical_is_absolute (index), canonical_resolved (index), canonical_issues (index), canonical_verdict (index), host_changed (index), protocol_upgraded (index), www_changed (index), trailing_slash_changed (index), case_changed (index), path_changed (index), params_changed (index), meta_refresh_present (index), meta_refresh_delay (index), meta_refresh_target (index), content_type_is_html (index), is_pdf (index), is_document_resource (index), charset_matches_header (index), charset_declared (index), viewport_present (index), viewport_blocks_scaling (index), is_mobile_friendly (index), html_lang_present (index), cookie_count (index), has_analytics_cookie (index), has_session_cookie (index)

Security list
Field details

Selector: css=self

Transformations:

  1. script (code: var it=(Array.isArray(value)&&value.length)?value[0]:{};var h=(it.headers&&typeof it.headers==="object")?it.headers:{};function g(n){var v=h[n];return (typeof v==="string")?v:"";}function lo(x){return String(x).toLowerCase();}var hsts=g("strict-transport-security");var hm=hsts.match(/max-age\s*=\s*(\d+)/i);var hstsDays=hm?Math.round(parseInt(hm[1],10)/86400):0;var csp=g("content-security-policy")||g("content-security-policy-report-only");var xfo=g("x-frame-options");var xcto=g("x-content-type-options");var rp=g("referrer-policy");var pp=g("permissions-policy")||g("feature-policy");var xxp=g("x-xss-protection");var coop=g("cross-origin-opener-policy");var coep=g("cross-origin-embedder-policy");var corp=g("cross-origin-resource-policy");var srv=g("server");var leak=/\d+\.\d+/.test(srv)?srv:"";var powered=g("x-powered-by")||g("x-aspnet-version")||g("x-aspnetmvc-version");var present=[],missing=[];if(hsts)present.push("HSTS");else missing.push("Strict-Transport-Security");if(csp)present.push("CSP");else missing.push("Content-Security-Policy");if(xfo)present.push("X-Frame-Options");else missing.push("X-Frame-Options");if(lo(xcto)==="nosniff")present.push("X-Content-Type-Options");else missing.push("X-Content-Type-Options");if(rp)present.push("Referrer-Policy");else missing.push("Referrer-Policy");if(pp)present.push("Permissions-Policy");else missing.push("Permissions-Policy");if(coop)present.push("COOP");else missing.push("Cross-Origin-Opener-Policy");var score=Math.round((present.length/7)*100);var grade=score>=90?"A":score>=75?"B":score>=55?"C":score>=35?"D":"F";var pageUrl=String(it.page_url||"");var isHttps=/^https:/i.test(pageUrl);return {hsts:hsts,hsts_max_age_days:String(hstsDays),hsts_includesubdomains:String(hsts.toLowerCase().indexOf("includesubdomains")!==-1),hsts_preload:String(hsts.toLowerCase().indexOf("preload")!==-1),csp:csp,csp_unsafe_inline:String(csp.indexOf("unsafe-inline")!==-1),csp_unsafe_eval:String(csp.indexOf("unsafe-eval")!==-1),x_frame_options:xfo,x_content_type_options:xcto,referrer_policy:rp,permissions_policy:pp,x_xss_protection:xxp,coop:coop,coep:coep,corp:corp,server_version_leak:leak,powered_by_leak:powered,headers_present:present.join(", "),headers_missing:missing.join(", "),security_score:String(score),security_grade:grade,mixed_content_risk:String(isHttps?"Check outlinks for http:// resources":"Page served over HTTP"),header_count:String(Object.keys(h).length)};)

Nested list fields: headers (text), page_url (page_url), hsts (index), hsts_max_age_days (index), hsts_includesubdomains (index), hsts_preload (index), csp (index), csp_unsafe_inline (index), csp_unsafe_eval (index), x_frame_options (index), x_content_type_options (index), referrer_policy (index), permissions_policy (index), x_xss_protection (index), coop (index), coep (index), corp (index), server_version_leak (index), powered_by_leak (index), headers_present (index), headers_missing (index), security_score (index), security_grade (index), mixed_content_risk (index), header_count (index)

OnPage_Elements

Item selector: css=html - Required: Page_URL

Fields of dataset "OnPage_Elements"
FieldProperty typeSelector / notesDefault
Page_URL page_url
Title text
Field details

Selector: css=head title

Transformations:

  1. trim

Title_Count css=head title count
Meta_Description attribute
Field details

Selector: css=meta[name='description']

Attribute: content

Meta_Description_Count css=meta[name='description'] count
Meta_Keywords attribute
Field details

Selector: css=meta[name='keywords']

Attribute: content

Meta_Author attribute
Field details

Selector: css=meta[name='author']

Attribute: content

Meta_Robots css=meta[name='robots'] text
Meta_Google_Site_Verification attribute
Field details

Selector: css=meta[name='google-site-verification']

Attribute: content

Generator attribute
Field details

Selector: css=meta[name='generator']

Attribute: content

Favicon attribute
Field details

Selector: css=link[rel='icon'],link[rel='shortcut icon'],link[rel='apple-touch-icon']

Attribute: href

Favicon_Count css=link[rel='icon'],link[rel='shortcut icon'],link[rel='apple-touch-icon'] count
Theme_Color attribute
Field details

Selector: css=meta[name='theme-color']

Attribute: content

Manifest attribute
Field details

Selector: css=link[rel='manifest']

Attribute: href

AMP_Link attribute
Field details

Selector: css=link[rel='amphtml']

Attribute: href

Is_AMP_Page css=html[amp],html[âš¡] count
Rel_Next attribute
Field details

Selector: css=link[rel='next']

Attribute: href

Rel_Prev attribute
Field details

Selector: css=link[rel='prev']

Attribute: href

Total_Meta_Tags css=head meta:not([name^='rtila-']) count
Total_Link_Tags css=head link count
Title_Chars text
Field details

Selector: css=head title

Transformations:

  1. script (code: return String(String(value==null?'':value).trim().length);, default_value: 0)

Title_Words text
Field details

Selector: css=head title

Transformations:

  1. script (code: var t=String(value==null?'':value).trim();return String(t?t.split(/\s+/).length:0);, default_value: 0)

Title_Pixels text
Field details

Selector: css=head title

Transformations:

  1. script (code: var W={" ":278,"!":278,"\"":355,"#":556,"$":556,"%":889,"&":667,"'":191,"(":333,")":333,"*":389,"+":584,",":278,"-":333,".":278,"/":278,":":278,";":278,"<":584,"=":584,">":584,"?":556,"@":1015,"[":278,"\\":278,"]":278,"^":469,"_":556,"`":333,"{":334,"|":260,"}":334,"~":584,"A":667,"B":667,"C":722,"D":722,"E":667,"F":611,"G":778,"H":722,"I":278,"J":500,"K":667,"L":556,"M":833,"N":722,"O":778,"P":667,"Q":778,"R":722,"S":667,"T":611,"U":722,"V":667,"W":944,"X":667,"Y":667,"Z":611,"a":556,"b":556,"c":500,"d":556,"e":556,"f":278,"g":556,"h":556,"i":222,"j":222,"k":500,"l":222,"m":833,"n":556,"o":556,"p":556,"q":556,"r":333,"s":500,"t":278,"u":556,"v":500,"w":722,"x":500,"y":500,"z":500}; for(var d=0;d<=9;d++){W[String(d)]=556;} var t=String(value==null?"":value).trim(); var u=0; for(var i=0;i<t.length;i++){u+=(W[t.charAt(i)]||600);} return String(Math.round(u*20/1000));, default_value: 0)

Title_SERP_Truncated text
Field details

Selector: css=head title

Transformations:

  1. script (code: var W={" ":278,"!":278,"\"":355,"#":556,"$":556,"%":889,"&":667,"'":191,"(":333,")":333,"*":389,"+":584,",":278,"-":333,".":278,"/":278,":":278,";":278,"<":584,"=":584,">":584,"?":556,"@":1015,"[":278,"\\":278,"]":278,"^":469,"_":556,"`":333,"{":334,"|":260,"}":334,"~":584,"A":667,"B":667,"C":722,"D":722,"E":667,"F":611,"G":778,"H":722,"I":278,"J":500,"K":667,"L":556,"M":833,"N":722,"O":778,"P":667,"Q":778,"R":722,"S":667,"T":611,"U":722,"V":667,"W":944,"X":667,"Y":667,"Z":611,"a":556,"b":556,"c":500,"d":556,"e":556,"f":278,"g":556,"h":556,"i":222,"j":222,"k":500,"l":222,"m":833,"n":556,"o":556,"p":556,"q":556,"r":333,"s":500,"t":278,"u":556,"v":500,"w":722,"x":500,"y":500,"z":500}; for(var d=0;d<=9;d++){W[String(d)]=556;} var t=String(value==null?"":value).trim(); var u=0; for(var i=0;i<t.length;i++){u+=(W[t.charAt(i)]||600);} return String(Math.round(u*20/1000) > 580);, default_value: false)

Title_Verdict text
Field details

Selector: css=head title

Transformations:

  1. script (code: var t=String(value==null?'':value).trim();if(!t)return 'MISSING';var L=t.length;if(L<10)return 'TOO SHORT';if(L>60)return 'TOO LONG';return 'OK';, default_value: MISSING)

Description_Chars attribute
Field details

Selector: css=meta[name='description']

Transformations:

  1. script (code: return String(String(value==null?'':value).trim().length);, default_value: 0)

Attribute: content

Description_Words attribute
Field details

Selector: css=meta[name='description']

Transformations:

  1. script (code: var t=String(value==null?'':value).trim();return String(t?t.split(/\s+/).length:0);, default_value: 0)

Attribute: content

Description_Pixels attribute
Field details

Selector: css=meta[name='description']

Transformations:

  1. script (code: var W={" ":278,"!":278,"\"":355,"#":556,"$":556,"%":889,"&":667,"'":191,"(":333,")":333,"*":389,"+":584,",":278,"-":333,".":278,"/":278,":":278,";":278,"<":584,"=":584,">":584,"?":556,"@":1015,"[":278,"\\":278,"]":278,"^":469,"_":556,"`":333,"{":334,"|":260,"}":334,"~":584,"A":667,"B":667,"C":722,"D":722,"E":667,"F":611,"G":778,"H":722,"I":278,"J":500,"K":667,"L":556,"M":833,"N":722,"O":778,"P":667,"Q":778,"R":722,"S":667,"T":611,"U":722,"V":667,"W":944,"X":667,"Y":667,"Z":611,"a":556,"b":556,"c":500,"d":556,"e":556,"f":278,"g":556,"h":556,"i":222,"j":222,"k":500,"l":222,"m":833,"n":556,"o":556,"p":556,"q":556,"r":333,"s":500,"t":278,"u":556,"v":500,"w":722,"x":500,"y":500,"z":500}; for(var d=0;d<=9;d++){W[String(d)]=556;} var t=String(value==null?"":value).trim(); var u=0; for(var i=0;i<t.length;i++){u+=(W[t.charAt(i)]||600);} return String(Math.round(u*14/1000));, default_value: 0)

Attribute: content

Description_SERP_Truncated attribute
Field details

Selector: css=meta[name='description']

Transformations:

  1. script (code: var W={" ":278,"!":278,"\"":355,"#":556,"$":556,"%":889,"&":667,"'":191,"(":333,")":333,"*":389,"+":584,",":278,"-":333,".":278,"/":278,":":278,";":278,"<":584,"=":584,">":584,"?":556,"@":1015,"[":278,"\\":278,"]":278,"^":469,"_":556,"`":333,"{":334,"|":260,"}":334,"~":584,"A":667,"B":667,"C":722,"D":722,"E":667,"F":611,"G":778,"H":722,"I":278,"J":500,"K":667,"L":556,"M":833,"N":722,"O":778,"P":667,"Q":778,"R":722,"S":667,"T":611,"U":722,"V":667,"W":944,"X":667,"Y":667,"Z":611,"a":556,"b":556,"c":500,"d":556,"e":556,"f":278,"g":556,"h":556,"i":222,"j":222,"k":500,"l":222,"m":833,"n":556,"o":556,"p":556,"q":556,"r":333,"s":500,"t":278,"u":556,"v":500,"w":722,"x":500,"y":500,"z":500}; for(var d=0;d<=9;d++){W[String(d)]=556;} var t=String(value==null?"":value).trim(); var u=0; for(var i=0;i<t.length;i++){u+=(W[t.charAt(i)]||600);} return String(Math.round(u*14/1000) > 990);, default_value: false)

Attribute: content

Description_Verdict attribute
Field details

Selector: css=meta[name='description']

Transformations:

  1. script (code: var t=String(value==null?'':value).trim();if(!t)return 'MISSING';var L=t.length;if(L<50)return 'TOO SHORT';if(L>160)return 'TOO LONG';return 'OK';, default_value: MISSING)

Attribute: content

H1_Count css=h1 count
H1_Text text
Field details

Selector: css=h1

Transformations:

  1. trim

H2_Count css=h2 count
H3_Count css=h3 count
H4_Count css=h4 count
H5_Count css=h5 count
H6_Count css=h6 count
Heading_Count_Total css=h1,h2,h3,h4,h5,h6 count
Hreflang_Count css=link[rel='alternate'][hreflang] count
Hreflang_Raw attribute
Field details

Selector: css=link[rel='alternate'][hreflang]

Attribute: hreflang

Heading_Structure list
Field details

Selector: css=h1,h2,h3,h4,h5,h6

Transformations:

  1. script (code: var rows=Array.isArray(value)?value:[];var counts={1:0,2:0,3:0,4:0,5:0,6:0};var outline=[],orderIssues=[],emptyCount=0,longCount=0,seen={},dupes=[];var h1First="",headingWords=0,firstLevel=0,maxLevel=0;for(var i=0;i<rows.length;i++){var tag=String(rows[i].tag||"").toLowerCase();var lvl=parseInt(tag.replace("h",""),10);if(!isFinite(lvl)||lvl<1||lvl>6)continue;counts[lvl]++;if(!firstLevel)firstLevel=lvl;if(lvl>maxLevel)maxLevel=lvl;var txt=String(rows[i].text||"").replace(/\s+/g," ").trim();if(txt==="")emptyCount++;if(txt.length>120)longCount++;headingWords+=txt?txt.split(/\s+/).length:0;outline.push(tag.toUpperCase()+": "+(txt.substring(0,90)||(txt?"":"(empty)")));if(seen[txt]&&txt!=="")dupes.push(txt.substring(0,60));else seen[txt]=1;if(i>0){var prev=parseInt(String(rows[i-1].tag||"h0").toLowerCase().replace("h",""),10);if(isFinite(prev)&&lvl-prev>1)orderIssues.push("H"+prev+" -> H"+lvl+" at #"+(i+1));}}var h1Texts=[];for(var j=0;j<rows.length;j++){if(String(rows[j].tag||"").toLowerCase()==="h1"){var t2=String(rows[j].text||"").replace(/\s+/g," ").trim();if(t2)h1Texts.push(t2);}}h1First=h1Texts[0]||"";var verdict=[];if(counts[1]===0)verdict.push("Missing H1");if(counts[1]>1)verdict.push("Multiple H1 ("+counts[1]+")");if(orderIssues.length)verdict.push(orderIssues.length+" order violation(s)");if(emptyCount)verdict.push(emptyCount+" empty heading(s)");if(dupes.length)verdict.push(dupes.length+" duplicate heading(s)");return {outline:outline.join(" | ").substring(0,8000),h1_count:String(counts[1]),h1_missing:String(counts[1]===0),h1_multiple:String(counts[1]>1),h1_first_text:h1First,h1_chars:String(h1First.length),h2_texts:"",order_violations:String(orderIssues.length),order_violation_list:orderIssues.join("; "),empty_headings:String(emptyCount),duplicate_headings:String(dupes.length),long_headings:String(longCount),heading_word_total:String(headingWords),first_heading_level:firstLevel?String(firstLevel):"",max_heading_level:String(maxLevel),headings_verdict:verdict.length?verdict.join("; "):"OK"};)

Nested list fields: tag (property), text (text), outline (index), h1_count (index), h1_missing (index), h1_multiple (index), h1_first_text (index), h1_chars (index), h2_texts (index), order_violations (index), order_violation_list (index), empty_headings (index), duplicate_headings (index), long_headings (index), heading_word_total (index), first_heading_level (index), max_heading_level (index), headings_verdict (index)

Hreflang_Audit list
Field details

Selector: css=link[rel='alternate'][hreflang]

Transformations:

  1. script (code: var rows=Array.isArray(value)?value:[];var pageUrl=rows.length?String(rows[0].page_url||""):";var pu=null;try{pu=new URL(pageUrl);}catch(e){}var vals=[],langs=[],regions=[],relative=0,dupes={},dupeList=[],selfRef=false,hasDefault=false;for(var i=0;i<rows.length;i++){var l=String(rows[i].lang||"").trim();var hr=String(rows[i].href||"").trim();if(l)vals.push(l+":"+hr);if(l.toLowerCase()==="x-default")hasDefault=true;var lp=l.split("-")[0].toLowerCase();if(lp&&langs.indexOf(lp)===-1)langs.push(lp);if(l.indexOf("-")!==-1){var rg=l.split("-")[1].toUpperCase();if(regions.indexOf(rg)===-1)regions.push(rg);}if(hr&&!/^https?:\/\//i.test(hr))relative++;if(dupes[l])dupeList.push(l);else dupes[l]=1;if(hr&&pu){try{var hu=new URL(hr,pageUrl);if(hu.origin===pu.origin&&hu.pathname===pu.pathname&&hu.search===pu.search)selfRef=true;}catch(e){}}}var issues=[];if(rows.length===0)issues.push("No hreflang annotations");else{if(!selfRef)issues.push("Missing self-referencing hreflang");if(!hasDefault)issues.push("Missing x-default");if(relative>0)issues.push(relative+" relative hreflang URL(s)");if(dupeList.length)issues.push("Duplicate hreflang values: "+dupeList.join(", "));}return {count:String(rows.length),values:vals.join(" | ").substring(0,4000),languages:langs.join(", "),regions:regions.join(", "),has_x_default:String(hasDefault),self_referencing:String(selfRef),relative_urls:String(relative),duplicate_langs:dupeList.join(", "),issues:issues.join("; ")};)

Nested list fields: lang (attribute), href (attribute), page_url (page_url), count (index), values (index), languages (index), regions (index), has_x_default (index), self_referencing (index), relative_urls (index), duplicate_langs (index), issues (index)

Social_Meta list
Field details

Selector: css=self

Transformations:

  1. script (code: var it=(Array.isArray(value)&&value.length)?value[0]:{};var head=String(it.head_html||"");var pageUrl=String(it.page_url||"");function grab(re){var m=head.match(re);return m?m[1].replace(/&amp;/g,"&").trim():"";}var ogTitle=grab(/property=["']og:title["'][^>]*content=["']([^"']*)["']/i)||grab(/content=["']([^"']*)["'][^>]*property=["']og:title["']/i);var ogDesc=grab(/property=["']og:description["'][^>]*content=["']([^"']*)["']/i)||grab(/content=["']([^"']*)["'][^>]*property=["']og:description["']/i);var ogImage=grab(/property=["']og:image["'][^>]*content=["']([^"']*)["']/i)||grab(/content=["']([^"']*)["'][^>]*property=["']og:image["']/i);var ogImageSecure=grab(/property=["']og:image:secure_url["'][^>]*content=["']([^"']*)["']/i);var ogW=grab(/property=["']og:image:width["'][^>]*content=["']([^"']*)["']/i);var ogH=grab(/property=["']og:image:height["'][^>]*content=["']([^"']*)["']/i);var ogAlt=grab(/property=["']og:image:alt["'][^>]*content=["']([^"']*)["']/i);var ogUrl=grab(/property=["']og:url["'][^>]*content=["']([^"']*)["']/i);var ogType=grab(/property=["']og:type["'][^>]*content=["']([^"']*)["']/i);var ogSite=grab(/property=["']og:site_name["'][^>]*content=["']([^"']*)["']/i);var ogLocale=grab(/property=["']og:locale["'][^>]*content=["']([^"']*)["']/i);var twCard=grab(/name=["']twitter:card["'][^>]*content=["']([^"']*)["']/i);var twTitle=grab(/name=["']twitter:title["'][^>]*content=["']([^"']*)["']/i);var twDesc=grab(/name=["']twitter:description["'][^>]*content=["']([^"']*)["']/i);var twImage=grab(/name=["']twitter:image["'][^>]*content=["']([^"']*)["']/i);var twSite=grab(/name=["']twitter:site["'][^>]*content=["']([^"']*)["']/i);var twCreator=grab(/name=["']twitter:creator["'][^>]*content=["']([^"']*)["']/i);var artPub=grab(/property=["']article:published_time["'][^>]*content=["']([^"']*)["']/i);var artMod=grab(/property=["']article:modified_time["'][^>]*content=["']([^"']*)["']/i);var artAuthor=grab(/property=["']article:author["'][^>]*content=["']([^"']*)["']/i);var artSection=grab(/property=["']article:section["'][^>]*content=["']([^"']*)["']/i);var ogCount=(head.match(/property=["']og:/gi)||[]).length;var twCount=(head.match(/name=["']twitter:/gi)||[]).length;var artTags=[];var re=/property=["']article:tag["'][^>]*content=["']([^"']*)["']/gi;var mm;while((mm=re.exec(head))!==null){artTags.push(mm[1]);}var ogMissing=[];if(!ogTitle)ogMissing.push("og:title");if(!ogDesc)ogMissing.push("og:description");if(!ogImage)ogMissing.push("og:image");if(!ogUrl)ogMissing.push("og:url");if(!ogType)ogMissing.push("og:type");var twMissing=[];if(!twCard)twMissing.push("twitter:card");if(!twTitle)twMissing.push("twitter:title");if(!twDesc)twMissing.push("twitter:description");if(!twImage)twMissing.push("twitter:image");var ogUrlMatch=false;if(ogUrl&&pageUrl){try{var a=new URL(ogUrl),b=new URL(pageUrl);ogUrlMatch=(a.origin===b.origin&&a.pathname===b.pathname);}catch(e){}}var ogImgAbs=/^https?:\/\//i.test(ogImage);var score=0,total=9;if(ogTitle)score++;if(ogDesc)score++;if(ogImage)score++;if(ogUrl)score++;if(ogType)score++;if(twCard)score++;if(twTitle)score++;if(twDesc)score++;if(twImage)score++;var pct=Math.round((score/total)*100);var verdict=pct>=90?"Excellent":pct>=70?"Good":pct>=40?"Incomplete":"Missing";return {og_title:ogTitle,og_description:ogDesc,og_image:ogImage,og_image_secure:ogImageSecure,og_image_width:ogW,og_image_height:ogH,og_image_alt:ogAlt,og_url:ogUrl,og_type:ogType,og_site_name:ogSite,og_locale:ogLocale,og_count:String(ogCount),og_missing:ogMissing.join(", "),twitter_card:twCard,twitter_title:twTitle,twitter_description:twDesc,twitter_image:twImage,twitter_site:twSite,twitter_creator:twCreator,twitter_count:String(twCount),twitter_missing:twMissing.join(", "),article_published:artPub,article_modified:artMod,article_author:artAuthor,article_section:artSection,article_tags:artTags.join(", "),og_url_matches_page:String(ogUrlMatch),og_image_is_absolute:String(ogImgAbs),social_score:String(pct),social_verdict:verdict};)

Nested list fields: head_html (html), page_url (page_url), og_title (index), og_description (index), og_image (index), og_image_secure (index), og_image_width (index), og_image_height (index), og_image_alt (index), og_url (index), og_type (index), og_site_name (index), og_locale (index), og_count (index), og_missing (index), twitter_card (index), twitter_title (index), twitter_description (index), twitter_image (index), twitter_site (index), twitter_creator (index), twitter_count (index), twitter_missing (index), article_published (index), article_modified (index), article_author (index), article_section (index), article_tags (index), og_url_matches_page (index), og_image_is_absolute (index), social_score (index), social_verdict (index)

Content_Readability

Item selector: css=html - Required: Page_URL

Fields of dataset "Content_Readability"
FieldProperty typeSelector / notesDefault
Page_URL page_url
Paragraph_Count css=body p count
List_Count css=body ul,ol count
Table_Count css=body table count
Blockquote_Count css=body blockquote count
Main_Landmark css=main,[role='main'],article count
Content_Metrics list
Field details

Selector: css=self

Transformations:

  1. script (code: var it=(Array.isArray(value)&&value.length)?value[0]:{};var text=String(it.text||"").replace(/[\u200B-\u200D\uFEFF]/g," ").replace(/\s+/g," ").trim();var html=String(it.html||"");var title=String(it.title||"").replace(/\s+/g," ").trim();var h1=String(it.h1||"").replace(/\s+/g," ").trim();var desc=String(it.desc||"").replace(/\s+/g," ").trim();var STOP={the:1,be:1,to:1,of:1,and:1,a:1,in:1,that:1,have:1,i:1,it:1,for:1,not:1,on:1,with:1,he:1,as:1,you:1,do:1,at:1,this:1,but:1,his:1,by:1,from:1,they:1,we:1,say:1,her:1,she:1,or:1,an:1,will:1,my:1,one:1,all:1,would:1,there:1,their:1,what:1,so:1,up:1,out:1,if:1,about:1,who:1,get:1,which:1,go:1,me:1,when:1,make:1,can:1,like:1,time:1,no:1,just:1,him:1,know:1,take:1,people:1,into:1,year:1,your:1,good:1,some:1,could:1,them:1,see:1,other:1,than:1,then:1,now:1,look:1,only:1,come:1,its:1,over:1,think:1,also:1,back:1,after:1,use:1,two:1,how:1,our:1,work:1,first:1,well:1,way:1,even:1,new:1,want:1,because:1,any:1,these:1,give:1,day:1,most:1,us:1,is:1,are:1,was:1,were:1,been:1,has:1,had:1,did:1,does:1};var words=text?text.split(" ").filter(function(w){return w.length>0;}):[];var wc=words.length;var sentences=text.split(/[.!?]+(?:\s|$)/).filter(function(s){return s.trim().length>1;});var sc=sentences.length||1;function syl(w){w=w.toLowerCase().replace(/[^a-z]/g,"");if(!w)return 0;if(w.length<=3)return 1;w=w.replace(/(?:[^laeiouy]es|ed|[^laeiouy]e)$/,"").replace(/^y/,"");var m=w.match(/[aeiouy]{1,2}/g);return m?m.length:1;}var sylTotal=0,longWords=0,chars=0;var uniq={},topMap={};for(var i=0;i<words.length;i++){var raw=words[i];var w=raw.toLowerCase().replace(/[^a-z0-9'-]/g,"");if(!w)continue;chars+=w.length;var sy=syl(w);sylTotal+=sy;if(sy>=3)longWords++;uniq[w]=1;if(w.length>2&&!STOP[w])topMap[w]=(topMap[w]||0)+1;}var uw=Object.keys(uniq).length;var asl=wc/sc;var asw=wc?sylTotal/wc:0;function r2(x){return Math.round(x*100)/100;}var fre=wc?r2(206.835-(1.015*asl)-(84.6*asw)):0;var fkg=wc?r2((0.39*asl)+(11.8*asw)-15.59):0;var fog=wc?r2(0.4*(asl+(wc?(longWords/wc)*100:0))):0;var smog=sc>=30?r2(1.0430*Math.sqrt((longWords*(30/sc)))+3.1291):0;var ari=wc?r2((4.71*(chars/wc))+(0.5*(wc/sc))-21.43):0;var cl=wc?r2((0.0588*(chars/wc*100)/100)*100-(0.296*(wc/sc))-15.8):0;var readVerdict=fre>=90?"Very Easy":fre>=80?"Easy":fre>=70?"Fairly Easy":fre>=60?"Plain":fre>=50?"Fairly Difficult":fre>=30?"Difficult":wc?"Very Difficult":"No text";var textChars=text.length,htmlChars=html.length;var ratio=htmlChars?r2((textChars/htmlChars)*100):0;var ratioVerdict=ratio===0?"No text":ratio<10?"Very Low":ratio<25?"Low":ratio<70?"Healthy":"High";var readMin=Math.max(1,Math.round(wc/225));var pairs=Object.keys(topMap).map(function(k){return [k,topMap[k]];});pairs.sort(function(a,b){return b[1]-a[1];});var top=pairs.slice(0,15).map(function(p){return p[0]+"("+p[1]+")";});var stopCount=0;for(var j=0;j<words.length;j++){var lw=words[j].toLowerCase().replace(/[^a-z]/g,"");if(STOP[lw])stopCount++;}var stopRatio=wc?r2((stopCount/wc)*100):0;var longest=0;for(var k2=0;k2<words.length;k2++){if(words[k2].length>longest)longest=words[k2].length;}var avgWordLen=wc?r2(chars/wc):0;function cyrb128(str){var h1=1779033703,h2=3144134277,h3=1013904242,h4=2773480762;for(var q=0,chr;q<str.length;q++){chr=str.charCodeAt(q);h1=Math.imul(h1^chr,2654435761);h2=Math.imul(h2^chr,1597334677);h3=Math.imul(h3^chr,2246822507);h4=Math.imul(h4^chr,3266489909);}h1=Math.imul(h1^(h1>>>16),2246822507)^Math.imul(h2^(h2>>>13),3266489909);h2=Math.imul(h2^(h2>>>16),2246822507)^Math.imul(h1^(h1>>>13),3266489909);h3=Math.imul(h3^(h3>>>16),2246822507)^Math.imul(h4^(h4>>>13),3266489909);h4=Math.imul(h4^(h4>>>16),2246822507)^Math.imul(h3^(h3>>>13),3266489909);return [h1>>>0,h2>>>0,h3>>>0,h4>>>0];}var norm=text.toLowerCase().replace(/[^a-z0-9\s]/g," ").replace(/\s+/g," ").trim();var shingles={};var nWords=norm.split(" ");for(var s2=0;s2+2<nWords.length;s2++){var sh=nWords[s2]+" "+nWords[s2+1]+" "+nWords[s2+2];shingles[sh]=(shingles[sh]||0)+1;}var shKeys=Object.keys(shingles);var sim=0;if(shKeys.length){var bits=new Array(64);for(var b=0;b<64;b++)bits[b]=0;for(var z=0;z<shKeys.length;z++){var hh=cyrb128(shKeys[z]);var hex="";for(var y=0;y<4;y++){hex+=("00000000"+hh[y].toString(16)).slice(-8);}for(var bb=0;bb<64;bb++){bits[bb]+=(hex.charAt(bb)==="1"?1:-1)*shingles[shKeys[z]];}}for(var b2=0;b2<64;b2++){if(bits[b2]>0)sim=sim*2+1;else sim=sim*2;}sim=sim>>>0;}var hashArr=cyrb128(norm);var hashHex="";for(var w2=0;w2<4;w2++){hashHex+=("00000000"+hashArr[w2].toString(16)).slice(-8);}var simHex=("0000000000000000"+sim.toString(16)).slice(-16);var langGuess="";var hl=String(it.html_lang||"").trim();if(hl)langGuess=hl;else{var enScore=(text.match(/\b(the|and|of|to|in|is|for|with)\b/gi)||[]).length;var esScore=(text.match(/\b(el|la|de|que|y|en|un|una|los|las)\b/gi)||[]).length;if(enScore>esScore&&enScore>5)langGuess="en";else if(esScore>5)langGuess="es";else langGuess="unknown";}function norm2(x){return x.toLowerCase().replace(/[^a-z0-9]/g,"");}var titleInBody=norm2(title).length>6&&norm.indexOf(norm2(title))!==-1;var h1InBody=norm2(h1).length>3&&norm.indexOf(norm2(h1))!==-1;var descInBody=norm2(desc).length>20&&norm.indexOf(norm2(desc))!==-1;var lorem=/lorem\s+ipsum/i.test(text);var placeholder=/\b(coming soon|under construction|todo:|tbd|placeholder text|your text here)\b/i.test(text);var cy=null;var cym=text.match(/©|\(c\)|copyright/gi)?text.match(/(?:©|\(c\)|copyright)\s*(?:\d{4}\s*[-–]\s*)?(\d{4})/i):null;if(cym)cy=parseInt(cym[1],10);var nowYear=new Date().getFullYear();var cyAge=(cy&&cy>1990&&cy<=nowYear+1)?(nowYear-cy):null;var dateMatches=text.match(/\b(?:19|20)\d{2}-\d{2}-\d{2}\b|\b(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)[a-z]*\.?[\s]+\d{1,2},?[\s]+(?:19|20)\d{2}\b|\b\d{1,2}\s+(?:January|February|March|April|May|June|July|August|September|October|November|December)\s+(?:19|20)\d{2}\b/gi)||[];var latest="";for(var d2=0;d2<dateMatches.length;d2++){var dm=dateMatches[d2].match(/(19|20)\d{2}/);if(dm){var yr=parseInt(dm[0],10);if(yr>parseInt(latest||"0",10))latest=String(yr);}}var emails=[];var em=text.match(/[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g)||[];for(var e2=0;e2<em.length;e2++){if(emails.indexOf(em[e2])===-1)emails.push(em[e2]);print}var phones=[];var ph=text.match(/(?:\+?\d{1,3}[\s.-]?)?(?:\(\d{2,4}\)[\s.-]?)?\d{3}[\s.-]?\d{3,4}[\s.-]?\d{3,4}/g)||[];for(var p2=0;p2<ph.length&&phones.length<5;p2++){if(ph[p2].replace(/\D/g,"").length>=9&&phones.indexOf(ph[p2])===-1)phones.push(ph[p2]);}var socialRe=/(?:https?:\/\/)?(?:www\.)?(facebook|instagram|twitter|x|linkedin|youtube|pinterest|tiktok|github|reddit|whatsapp|t\.me|telegram)\.com\/[^\s"'<>)]+/gi;var socialMatches=html.match(socialRe)||[];var socialDomains={};for(var sm2=0;sm2<socialMatches.length;sm2++){var dm2=socialMatches[sm2].match(/(?:facebook|instagram|twitter|x|linkedin|youtube|pinterest|tiktok|github|reddit|whatsapp|t\.me|telegram)\./i);if(dm2)socialDomains[dm2[0].replace(/\.$/,"").toLowerCase()]=1;}var comments=(html.match(/<!--[\s\S]*?-->/g)||[]).length;var sensitive=/<!--[^>]*(password|secret|api[_-]?key|token|debug|do not remove|todo:|fixme)/i.test(html);var stuffScore=0;if(pairs.length&&wc>50){var topFreq=pairs[0][1];var density=(topFreq/wc)*100;stuffScore=Math.round(density*100)/100;}var stuffVerdict=stuffScore>8?"Keyword stuffing suspected":stuffScore>5?"High density":"Normal";var thin=wc<300;var lenClass=wc===0?"Empty":wc<150?"Very thin":wc<300?"Thin":wc<800?"Short":wc<2000?"Standard":"Long-form";return {word_count:String(wc),unique_word_count:String(uw),unique_word_ratio:String(wc?r2((uw/wc)*100):0),sentence_count:String(sc),avg_words_per_sentence:String(r2(asl)),syllable_count:String(sylTotal),avg_syllables_per_word:String(r2(asw)),flesch_reading_ease:String(fre),flesch_kincaid_grade:String(fkg),gunning_fog:String(fog),smog_index:String(smog),automated_readability:String(ari),coleman_liau:String(cl),readability_verdict:readVerdict,text_chars:String(textChars),html_chars:String(htmlChars),text_to_html_ratio:String(ratio),text_ratio_verdict:ratioVerdict,reading_time_min:String(readMin),top_words:top.join(", "),stopword_ratio:String(stopRatio),longest_word:String(longest),avg_word_length:String(avgWordLen),content_hash:hashHex,content_simhash:simHex,near_duplicate_key:simHex.substring(0,8),content_thin:String(thin),content_length_class:lenClass,language_guess:langGuess,title_in_body:String(titleInBody),h1_in_body:String(h1InBody),description_in_body:String(descInBody),has_lorem_ipsum:String(lorem),has_placeholder_text:String(placeholder),copyright_year:cy?String(cy):"",copyright_age_years:cyAge===null?"":String(cyAge),dates_found:String(dateMatches.length),latest_date_found:latest,emails_found:emails.slice(0,10).join(", "),phones_found:phones.join(", "),social_profile_count:String(Object.keys(socialDomains).length),social_profiles:Object.keys(socialDomains).join(", "),html_comment_count:String(comments),has_sensitive_comment:String(sensitive),keyword_stuffing_score:String(stuffScore),keyword_stuffing_verdict:stuffVerdict};)

Nested list fields: text (text), html (html), title (text), h1 (text), desc (attribute), html_lang (attribute), word_count (index), unique_word_count (index), unique_word_ratio (index), sentence_count (index), avg_words_per_sentence (index), syllable_count (index), avg_syllables_per_word (index), flesch_reading_ease (index), flesch_kincaid_grade (index), gunning_fog (index), smog_index (index), automated_readability (index), coleman_liau (index), readability_verdict (index), text_chars (index), html_chars (index), text_to_html_ratio (index), text_ratio_verdict (index), reading_time_min (index), top_words (index), stopword_ratio (index), longest_word (index), avg_word_length (index), content_hash (index), content_simhash (index), near_duplicate_key (index), content_thin (index), content_length_class (index), language_guess (index), title_in_body (index), h1_in_body (index), description_in_body (index), has_lorem_ipsum (index), has_placeholder_text (index), copyright_year (index), copyright_age_years (index), dates_found (index), latest_date_found (index), emails_found (index), phones_found (index), social_profile_count (index), social_profiles (index), html_comment_count (index), has_sensitive_comment (index), keyword_stuffing_score (index), keyword_stuffing_verdict (index)

Links_Census

Item selector: css=html - Required: Page_URL

Media_Performance

Item selector: css=html - Required: Page_URL

Fields of dataset "Media_Performance"
FieldProperty typeSelector / notesDefault
Page_URL page_url
DOM_Node_Count css=* count
Img_Count css=img count
Script_Count css=script count
Stylesheet_Count css=link[rel='stylesheet'] count
Iframe_Count css=iframe count
Form_Count css=form count
Input_Count css=input,textarea,select count
Button_Count css=button count
SVG_Count css=svg count
Noscript_Count css=noscript count
Inline_Style_Attrs css=[style] count
Image_Audit list
Field details

Selector: css=img

Transformations:

  1. script (code: var rows=Array.isArray(value)?value:[];var total=rows.length,missingAlt=0,emptyAlt=0,goodAlt=0,longAlt=0,missingSrc=0,emptySrc=0;var lazy=0,eager=0,jsLazy=0,hasSrcset=0,missingDims=0,hasDims=0,modern=0,legacy=0;var hasTitle=0,asyncDec=0,extImg=0,insecure=0;var fmt={},srcSeen={},altSamples=[];var pageUrl="",hostname="";try{pageUrl=String(rows[0].page_url||"");hostname=new URL(pageUrl).hostname;}catch(e){}for(var i=0;i<rows.length;i++){var r=rows[i]||{};var src=String(r.src||"").trim();var cs=String(r.current_src||"").trim();var eff=cs||src;var alt=r.alt;var hasAltAttr=(alt!==null&&alt!==undefined);if(!hasAltAttr){missingAlt++;if(altSamples.length<10)altSamples.push(eff.substring(eff.lastIndexOf("/")+1,eff.lastIndexOf("/")+60)||"(no src)");}else if(String(alt).trim()==="")emptyAlt++;else{goodAlt++;if(String(alt).length>125)longAlt++;}if(!hasAltAttr&&!src)missingSrc++;if(src==="")emptySrc++;if(String(r.loading||"").toLowerCase()==="lazy")lazy++;else if(src)eager++;if(!src&&String(r.data_src||"").trim())jsLazy++;if(String(r.srcset||"").trim())hasSrcset++;var w=String(r.w||"").trim(),h=String(r.h||"").trim();if(w&&h)hasDims++;else missingDims++;if(String(r.title||"").trim())hasTitle++;if(String(r.decoding||"").toLowerCase()==="async")asyncDec++;var ext="";var em=eff.match(/\.([a-z0-9]{2,5})(?:$|\?|#)/i);if(em)ext=em[1].toLowerCase();else if(eff.indexOf("data:image/")===0){var dm=eff.match(/^data:image\/([a-z0-9.+-]+)/i);if(dm)ext=dm[1].toLowerCase();}if(ext)fmt[ext]=(fmt[ext]||0)+1;if(ext==="webp"||ext==="avif"||ext==="jxl")modern++;else if(ext)legacy++;if(eff){var key=eff.split("?")[0];srcSeen[key]=(srcSeen[key]||0)+1;try{var u=new URL(eff,pageUrl);if(u.hostname&&u.hostname!==hostname)extImg++;if(u.protocol==="http:"&&/^https:/i.test(pageUrl))insecure++;}catch(e){}}}var dupes=0;Object.keys(srcSeen).forEach(function(k){if(srcSeen[k]>1)dupes+=srcSeen[k]-1;});var fmtStr=Object.keys(fmt).map(function(k){return k+"("+fmt[k]+")";}).sort().join(", ");var verdict=[];if(missingAlt>0)verdict.push(missingAlt+" img missing alt");if(missingDims>0&&total>0&&(missingDims/total)>0.3)verdict.push(Math.round((missingDims/total)*100)+"% missing width/height (CLS risk)");if(modern===0&&total>3)verdict.push("No modern image formats (WebP/AVIF)");if(insecure>0)verdict.push(insecure+" insecure http:// image(s)");return {total:String(total),missing_alt:String(missingAlt),empty_alt:String(emptyAlt),good_alt:String(goodAlt),long_alt:String(longAlt),alt_missing_sample:altSamples.join(" | "),missing_src:String(missingSrc),empty_src:String(emptySrc),lazy_loaded:String(lazy),eager_loaded:String(eager),js_lazy_loaded:String(jsLazy),has_srcset:String(hasSrcset),responsive_coverage:String(total?Math.round((hasSrcset/total)*100):0),missing_dimensions:String(missingDims),has_dimensions:String(hasDims),cls_risk_ratio:String(total?Math.round((missingDims/total)*100):0),modern_format:String(modern),legacy_format:String(legacy),format_breakdown:fmtStr,external_images:String(extImg),insecure_images:String(insecure),has_title_attr:String(hasTitle),async_decoding:String(asyncDec),unique_srcs:String(Object.keys(srcSeen).length),duplicate_srcs:String(dupes),image_verdict:verdict.length?verdict.join("; "):"OK"};)

Nested list fields: src (attribute), alt (attribute), w (attribute), h (attribute), loading (attribute), srcset (attribute), decoding (attribute), data_src (attribute), current_src (property), total (index), missing_alt (index), empty_alt (index), good_alt (index), long_alt (index), alt_missing_sample (index), missing_src (index), empty_src (index), lazy_loaded (index), eager_loaded (index), js_lazy_loaded (index), has_srcset (index), responsive_coverage (index), missing_dimensions (index), has_dimensions (index), cls_risk_ratio (index), modern_format (index), legacy_format (index), format_breakdown (index), external_images (index), insecure_images (index), has_title_attr (index), async_decoding (index), unique_srcs (index), duplicate_srcs (index), image_verdict (index)

Resource_Audit list
Field details

Selector: css=script,link[rel='stylesheet']

Transformations:

  1. script (code: var rows=Array.isArray(value)?value:[];var pageUrl="",hostname="";for(var q=0;q<rows.length;q++){if(rows[q]&&rows[q].page_url){pageUrl=String(rows[q].page_url);break;}}try{hostname=new URL(pageUrl).hostname;}catch(e){}var scripts=0,scriptsExt=0,scriptsInline=0,scriptsAsync=0,scriptsDefer=0,scriptsBlocking=0,scriptsThird=0,moduleScripts=0;var sheets=0,sheetsBlocking=0,sheetsPrint=0,sheetsThird=0;var thirdDomains={},insecure=0,blockSamples=[];for(var i=0;i<rows.length;i++){var r=rows[i]||{};var tag=String(r.tag||"").toLowerCase();var rel=String(r.rel||"").toLowerCase();var src=String(r.src||"").trim();var href=String(r.href||"").trim();var url=src||href;var third=false;if(url&&hostname){try{var u=new URL(url,pageUrl);if(u.hostname&&u.hostname!==hostname){third=true;thirdDomains[u.hostname]=(thirdDomains[u.hostname]||0)+1;}if(u.protocol==="http:"&&/^https:/i.test(pageUrl))insecure++;}catch(e){}}if(tag==="script"){scripts++;var type=String(r.type_a||"").toLowerCase();if(type.indexOf("module")!==-1)moduleScripts++;if(src){scriptsExt++;if(third)scriptsThird++;}else scriptsInline++;var isAsync=(r.async_a!==null&&r.async_a!==undefined)||(r.defer_a!==null&&r.defer_a!==undefined)||type==="module";if(isAsync){if(r.async_a!==null&&r.async_a!==undefined)scriptsAsync++;if(r.defer_a!==null&&r.defer_a!==undefined)scriptsDefer++;}else{scriptsBlocking++;if(blockSamples.length<12&&src)blockSamples.push(src);}}else if(tag==="link"&&rel.indexOf("stylesheet")!==-1){sheets++;var media=String(r.media_a||"").toLowerCase();if(media&&media!=="all"&&media.indexOf("print")!==-1)sheetsPrint++;else{sheetsBlocking++;if(blockSamples.length<12&&href)blockSamples.push(href);}if(third)sheetsThird++;}}var td=Object.keys(thirdDomains).map(function(k){return [k,thirdDomains[k]];});td.sort(function(a,b){return b[1]-a[1];});var total=rows.length;var blockingTotal=sheetsBlocking+scriptsBlocking;var verdict=blockingTotal===0?"No render-blocking resources":blockingTotal<=3?"Low render-blocking load ("+blockingTotal+")":blockingTotal<=8?"Moderate render-blocking load ("+blockingTotal+")":"Heavy render-blocking load ("+blockingTotal+")";return {total:String(total),scripts_total:String(scripts),scripts_external:String(scriptsExt),scripts_inline:String(scriptsInline),scripts_async:String(scriptsAsync),scripts_defer:String(scriptsDefer),scripts_blocking:String(scriptsBlocking),scripts_third_party:String(scriptsThird),third_party_domains:td.slice(0,20).map(function(p){return p[0]+"("+p[1]+")";}).join(", "),stylesheets_total:String(sheets),stylesheets_render_blocking:String(sheetsBlocking),stylesheets_print_only:String(sheetsPrint),stylesheets_third_party:String(sheetsThird),render_blocking_total:String(blockingTotal),render_blocking_sample:blockSamples.join(" | "),insecure_resources:String(insecure),module_scripts:String(moduleScripts),blocking_verdict:verdict};)

Nested list fields: tag (property), src (attribute), href (attribute), rel (attribute), async_a (attribute), defer_a (attribute), type_a (attribute), media_a (attribute), page_url (page_url), total (index), scripts_total (index), scripts_external (index), scripts_inline (index), scripts_async (index), scripts_defer (index), scripts_blocking (index), scripts_third_party (index), third_party_domains (index), stylesheets_total (index), stylesheets_render_blocking (index), stylesheets_print_only (index), stylesheets_third_party (index), render_blocking_total (index), render_blocking_sample (index), insecure_resources (index), module_scripts (index), blocking_verdict (index)

Embed_Audit list
Field details

Selector: css=iframe,video,audio,embed,object

Transformations:

  1. script (code: var rows=Array.isArray(value)?value:[];var total=rows.length,iframes=0,ifrNoTitle=0,ifrLazy=0,yt=0,vimeo=0,maps=0,ads=0,videos=0,audios=0,objects=0,insecure=0;var domains={};for(var i=0;i<rows.length;i++){var r=rows[i]||{};var tag=String(r.tag||"").toLowerCase();var src=String(r.src||"").trim();if(tag==="iframe"){iframes++;if(!String(r.title||"").trim())ifrNoTitle++;if(String(r.loading||"").toLowerCase()==="lazy")ifrLazy++;}else if(tag==="video")videos++;else if(tag==="audio")audios++;else if(tag==="embed"||tag==="object")objects++;if(src){try{var u=new URL(src,"about:blank");var h=u.hostname.replace(/^www\./,"");domains[h]=(domains[h]||0)+1;if(u.protocol==="http:")insecure++;if(/youtube\.com|youtu\.be/.test(h))yt++;else if(/vimeo\.com/.test(h))vimeo++;else if(/google\.[a-z.]+\/maps|maps\.google/.test(h))maps++;else if(/doubleclick|googlesyndication|adsystem|amazon-adsystem|adnxs|criteo|taboola|outbrain/.test(h))ads++;}catch(e){}}}return {total:String(total),iframes:String(iframes),iframes_missing_title:String(ifrNoTitle),iframes_lazy:String(ifrLazy),youtube_embeds:String(yt),vimeo_embeds:String(vimeo),map_embeds:String(maps),ad_embeds:String(ads),videos:String(videos),audios:String(audios),objects:String(objects),embed_domains:Object.keys(domains).map(function(k){return k+"("+domains[k]+")";}).join(", "),insecure_embeds:String(insecure)};)

Nested list fields: tag (property), src (attribute), title (attribute), loading (attribute), sandbox (attribute), total (index), iframes (index), iframes_missing_title (index), iframes_lazy (index), youtube_embeds (index), vimeo_embeds (index), map_embeds (index), ad_embeds (index), videos (index), audios (index), objects (index), embed_domains (index), insecure_embeds (index)

DOM_Performance list
Field details

Selector: css=self

Transformations:

  1. script (code: var it=(Array.isArray(value)&&value.length)?value[0]:{};var html=String(it.html||"");var domNodes=parseInt(it.dom_nodes,10)||0;var VOID={area:1,base:1,br:1,col:1,embed:1,hr:1,img:1,input:1,link:1,meta:1,param:1,source:1,track:1,wbr:1};var depth=0,maxDepth=0;var tagRe=/<(/)?([a-zA-Z][a-zA-Z0-9-]*)([^>]*)>/g;var m;while((m=tagRe.exec(html))!==null){var closing=m[1]==="/";var name=m[2].toLowerCase();var attrs=m[3]||"";if(name==="script"||name==="style"){if(!closing){var closeIdx=html.indexOf("</"+name,tagRe.lastIndex);if(closeIdx!==-1)tagRe.lastIndex=closeIdx;}continue;}if(VOID[name]||/\/\s*$/.test(attrs))continue;if(closing){if(depth>0)depth--;}else{depth++;if(depth>maxDepth)maxDepth=depth;}}var preload=parseInt(it.preload,10)||0;var prefetch=parseInt(it.prefetch,10)||0;var preconnect=parseInt(it.preconnect,10)||0;var modulepreload=parseInt(it.modulepreload,10)||0;var prerender=parseInt(it.prerender,10)||0;var fontPreload=parseInt(it.fonts_preload,10)||0;var inlineStyles=parseInt(it.inline_styles,10)||0;var styleAttrs=parseInt(it.style_attrs,10)||0;var hints=preload+prefetch+preconnect+modulepreload+prerender;var ttfb=parseInt(it.response_ms,10);if(!isFinite(ttfb))ttfb=0;var bytes=parseInt(it.content_bytes,10);if(!isFinite(bytes))bytes=0;var proto=String(it.protocol||"").toLowerCase();var domVerdict=domNodes===0?"Unknown":domNodes<=1500?"Healthy":domNodes<=3000?"Large":domNodes<=5000?"Very large":"Excessive";var depthVerdict=maxDepth<=15?"Healthy":maxDepth<=25?"Deep":"Excessively deep";var ttfbVerdict=ttfb===0?"Unknown":ttfb<=200?"Excellent (<=200ms)":ttfb<=600?"Good (<=600ms)":ttfb<=1800?"Needs improvement":"Poor (>1800ms)";var score=100;if(domNodes>1500)score-=Math.min(25,Math.floor((domNodes-1500)/400));if(maxDepth>20)score-=Math.min(15,Math.floor((maxDepth-20)/5));if(ttfb>600)score-=Math.min(25,Math.floor((ttfb-600)/200));if(bytes>500000)score-=Math.min(15,Math.floor(bytes/500000));if(hints===0)score-=5;if(score<0)score=0;var pVerdict=score>=85?"Excellent":score>=70?"Good":score>=50?"Needs improvement":"Poor";return {dom_size:String(domNodes),dom_size_verdict:domVerdict,dom_max_depth:String(maxDepth),dom_depth_verdict:depthVerdict,html_bytes:String(html.length),html_kb:String(Math.round(html.length/1024)),resource_hint_count:String(hints),preload_count:String(preload),prefetch_count:String(prefetch),preconnect_count:String(preconnect),font_preload_count:String(fontPreload),inline_style_blocks:String(inlineStyles),inline_style_attrs:String(styleAttrs),ttfb_ms:String(ttfb),ttfb_verdict:ttfbVerdict,transfer_kb:String(Math.round(bytes/1024)),uses_http2:String(proto==="h2"),uses_http3:String(proto==="h3"||proto==="h3-29"),performance_score:String(score),performance_verdict:pVerdict};)

Nested list fields: html (html), dom_nodes (count), preload (count), prefetch (count), preconnect (count), modulepreload (count), prerender (count), fonts_preload (count), inline_styles (count), style_attrs (count), canonical_tags (count), response_ms (text), content_bytes (text), protocol (text), dom_size (index), dom_size_verdict (index), dom_max_depth (index), dom_depth_verdict (index), html_bytes (index), html_kb (index), resource_hint_count (index), preload_count (index), prefetch_count (index), preconnect_count (index), font_preload_count (index), inline_style_blocks (index), inline_style_attrs (index), ttfb_ms (index), ttfb_verdict (index), transfer_kb (index), uses_http2 (index), uses_http3 (index), performance_score (index), performance_verdict (index)

Structured_Data

Item selector: css=html - Required: Page_URL

Fields of dataset "Structured_Data"
FieldProperty typeSelector / notesDefault
Page_URL page_url
JsonLD_Count css=script[type='application/ld+json'] count
Microdata_Count css=[itemtype] count
RDFa_Count css=[typeof] count
Tech_Stack text
Field details

Selector: variable=tech_stack

Transformations:

  1. script (code: if(!value)return '';if(Array.isArray(value))return value.join(', ');return String(value);, default_value: )

Tech_Stack_Count text
Field details

Selector: variable=tech_stack

Transformations:

  1. script (code: return String(Array.isArray(value)?value.length:0);, default_value: 0)

Structured_Data_Audit list
Field details

Selector: css=script[type='application/ld+json']

Transformations:

  1. script (code: var rows=Array.isArray(value)?value:[];var blocks=rows.length,valid=0,invalid=0,errors=[];var types={},ids=[],graphNodes=0,missingCtx=0,missingType=0;function walk(node,depth){if(!node||typeof node!=="object"||depth>8)return;if(Array.isArray(node)){for(var a=0;a<node.length;a++)walk(node[a],depth+1);return;}if(node["@graph"]&&Array.isArray(node["@graph"])){graphNodes+=node["@graph"].length;walk(node["@graph"],depth+1);}var t=node["@type"];if(t){if(Array.isArray(t)){for(var b=0;b<t.length;b++){var tn=String(t[b]);types[tn]=(types[tn]||0)+1;}}else{var tn2=String(t);types[tn2]=(types[tn2]||0)+1;}}else if(node["@id"]||node["@context"])missingType++;if(node["@id"])ids.push(String(node["@id"]).substring(0,120));if(node["@context"]===undefined&&node["@graph"]===undefined&&node["@type"]!==undefined)missingCtx++;var keys=Object.keys(node);for(var k=0;k<keys.length&&k<40;k++){var v=node[keys[k]];if(v&&typeof v==="object")walk(v,depth+1);}}for(var i=0;i<rows.length;i++){var raw=String(rows[i].json||"").trim();if(!raw){invalid++;errors.push("Block "+(i+1)+": empty");continue;}try{var parsed=JSON.parse(raw);valid++;walk(parsed,0);}catch(e){invalid++;errors.push("Block "+(i+1)+": "+String(e.message||e).substring(0,120));}}var typeNames=Object.keys(types);var has=function(re){for(var j=0;j<typeNames.length;j++){if(re.test(typeNames[j]))return true;}return false;};var rich=[];if(has(/^(Product|Offer|AggregateOffer)$/i))rich.push("Product/Merchant listing");if(has(/^(Article|NewsArticle|BlogPosting|TechArticle)$/i))rich.push("Article");if(has(/^FAQPage$/i))rich.push("FAQ");if(has(/^HowTo$/i))rich.push("HowTo");if(has(/^BreadcrumbList$/i))rich.push("Breadcrumb");if(has(/^(LocalBusiness|Restaurant|Store|Dentist|LegalService)$/i))rich.push("Local Business");if(has(/^(Review|AggregateRating)$/i))rich.push("Review stars");if(has(/^(Event)$/i))rich.push("Event");if(has(/^(VideoObject)$/i))rich.push("Video");if(has(/^(Recipe)$/i))rich.push("Recipe");if(has(/^(JobPosting)$/i))rich.push("Job posting");if(has(/^(SoftwareApplication)$/i))rich.push("Software app");if(has(/^(QAPage|Question|Answer)$/i))rich.push("Q&A");if(has(/^(SpeakableSpecification)$/i))rich.push("Speakable");if(has(/^(Organization)$/i))rich.push("Organization knowledge panel");if(has(/^(WebSite)$/i))rich.push("Sitelinks searchbox");var issues=[];if(blocks===0)issues.push("No JSON-LD found");if(invalid>0)issues.push(invalid+" invalid JSON-LD block(s)");if(missingCtx>0)issues.push(missingCtx+" node(s) missing @context");if(missingType>0)issues.push(missingType+" node(s) missing @type");var verdict=blocks===0?"Missing structured data":invalid>0?"Broken structured data":rich.length?"Rich result eligible: "+rich.join(", "):"Valid but no rich-result type";return {blocks:String(blocks),valid_blocks:String(valid),invalid_blocks:String(invalid),parse_errors:errors.join(" | ").substring(0,2000),types:typeNames.map(function(t){return t+"("+types[t]+")";}).join(", "),type_count:String(typeNames.length),has_organization:String(has(/^(Organization|Corporation|LocalBusiness)$/i)),has_localbusiness:String(has(/LocalBusiness|Restaurant|Store|Dentist|LegalService|AutoDealer|HealthAndBeautyBusiness/i)),has_website:String(has(/^WebSite$/i)),has_webpage:String(has(/^(WebPage|ItemPage|AboutPage|ContactPage|CollectionPage)$/i)),has_breadcrumb:String(has(/^BreadcrumbList$/i)),has_article:String(has(/^(Article|NewsArticle|BlogPosting|TechArticle|ScholarlyArticle)$/i)),has_product:String(has(/^(Product|Offer|AggregateOffer)$/i)),has_faq:String(has(/^FAQPage$/i)),has_howto:String(has(/^HowTo$/i)),has_review:String(has(/^Review$/i)),has_aggregate_rating:String(has(/^AggregateRating$/i)),has_video:String(has(/^(VideoObject)$/i)),has_event:String(has(/^(Event)$/i)),has_person:String(has(/^Person$/i)),has_speakable:String(has(/^(SpeakableSpecification)$/i)),missing_context:String(missingCtx),missing_type:String(missingType),graph_nodes:String(graphNodes),entity_ids:ids.slice(0,20).join(", "),rich_result_eligible:rich.join(", "),issues:issues.join("; "),structured_data_verdict:verdict};)

Nested list fields: json (text), page_url (page_url), blocks (index), valid_blocks (index), invalid_blocks (index), parse_errors (index), types (index), type_count (index), has_organization (index), has_localbusiness (index), has_website (index), has_webpage (index), has_breadcrumb (index), has_article (index), has_product (index), has_faq (index), has_howto (index), has_review (index), has_aggregate_rating (index), has_video (index), has_event (index), has_person (index), has_speakable (index), missing_context (index), missing_type (index), graph_nodes (index), entity_ids (index), rich_result_eligible (index), issues (index), structured_data_verdict (index)

Microdata_RDFa list
Field details

Selector: css=self

Transformations:

  1. script (code: var it=(Array.isArray(value)&&value.length)?value[0]:{};var itemtypes=parseInt(it.itemtypes,10)||0;var itemprops=parseInt(it.itemprops,10)||0;var typeofs=parseInt(it.typeof_nodes,10)||0;var props=parseInt(it.property_nodes,10)||0;var mdTypes=String(it.microdata_types||"").split("\n").map(function(s){return s.trim();}).filter(Boolean);var rdTypes=String(it.rdfa_types||"").split("\n").map(function(s){return s.trim();}).filter(Boolean);var mdClean=[],rdClean=[];for(var i=0;i<mdTypes.length;i++){var p=mdTypes[i].replace(/^https?:\/\/schema\.org\//i,"");if(mdClean.indexOf(p)===-1)mdClean.push(p);}for(var j=0;j<rdTypes.length;j++){var p2=rdTypes[j].replace(/^https?:\/\/schema\.org\//i,"");if(rdClean.indexOf(p2)===-1)rdClean.push(p2);}var formats=[];if(itemtypes>0||itemprops>0)formats.push("Microdata");if(typeofs>0||props>0)formats.push("RDFa");var verdict=formats.length===0?"No Microdata or RDFa markup":formats.join(" + ")+" detected";return {microdata_total:String(itemtypes),microdata_types_all:mdClean.join(", "),rdfa_total:String(typeofs),rdfa_types_all:rdClean.join(", "),markup_formats_used:formats.join(", "),markup_verdict:verdict};)

Nested list fields: itemtypes (count), itemprops (count), typeof_nodes (count), property_nodes (count), microdata_types (attribute), rdfa_types (attribute), microdata_total (index), microdata_types_all (index), rdfa_total (index), rdfa_types_all (index), markup_formats_used (index), markup_verdict (index)

Accessibility_AXE

Item selector: css=html - Required: Page_URL

Fields of dataset "Accessibility_AXE"
FieldProperty typeSelector / notesDefault
Page_URL page_url
AXE_Violations text
Field details

Selector: variable=axe_report

Transformations:

  1. json_path
  2. script (code: return String(Array.isArray(value)?value.length:0);, default_value: 0)

AXE_Passes text
Field details

Selector: variable=axe_report

Transformations:

  1. json_path
  2. script (code: return String(Array.isArray(value)?value.length:0);, default_value: 0)

AXE_Incomplete text
Field details

Selector: variable=axe_report

Transformations:

  1. json_path
  2. script (code: return String(Array.isArray(value)?value.length:0);, default_value: 0)

AXE_Inapplicable text
Field details

Selector: variable=axe_report

Transformations:

  1. json_path
  2. script (code: return String(Array.isArray(value)?value.length:0);, default_value: 0)

AXE_Summary list
Field details

Selector: css=self

Transformations:

  1. script (code: var it=(Array.isArray(value)&&value.length)?value[0]:{};var rep=it.report;if(!rep||typeof rep!=="object"){return {ran:"false",violation_count:"0",critical_count:"0",serious_count:"0",moderate_count:"0",minor_count:"0",affected_nodes:"0",violation_ids:"",top_violations:"",wcag_tags:"",passes_count:"0",incomplete_count:"0",axe_score:"",axe_grade:"",axe_verdict:"AXE did not run (blocked, CSP, or CDN unreachable)",axe_url:"",axe_engine_version:""};}var v=Array.isArray(rep.violations)?rep.violations:[];var p=Array.isArray(rep.passes)?rep.passes:[];var inc=Array.isArray(rep.incomplete)?rep.incomplete:[];var crit=0,ser=0,mod=0,min=0,nodes=0,ids=[],tops=[],tags={};for(var i=0;i<v.length;i++){var imp=String(v[i].impact||"").toLowerCase();var nc=Array.isArray(v[i].nodes)?v[i].nodes.length:0;nodes+=nc;if(imp==="critical")crit++;else if(imp==="serious")ser++;else if(imp==="moderate")mod++;else min++;ids.push(v[i].id);tops.push((v[i].id||"?")+" ["+(imp||"?")+":"+nc+"] "+String(v[i].help||"").substring(0,70));var t=Array.isArray(v[i].tags)?v[i].tags:[];for(var j=0;j<t.length;j++){if(/^wcag/i.test(t[j])||/^best-practice/i.test(t[j])||/^section508/i.test(t[j]))tags[t[j]]=1;}}tops.sort(function(a,b){var oa=["critical","serious","moderate","minor"];var ai=9,bi=9;for(var k=0;k<oa.length;k++){if(a.indexOf("["+oa[k])!==-1)ai=k;if(b.indexOf("["+oa[k])!==-1)bi=k;}return ai-bi;});var checked=p.length+v.length+inc.length;var score=checked?Math.round((p.length/checked)*100):100;var grade=score>=95?"A":score>=90?"B":score>=80?"C":score>=70?"D":"F";var verdict=(crit+ser)===0?(v.length===0?"No accessibility violations":v.length+" minor/moderate violation(s)"):(crit+" critical, "+ser+" serious violation(s)");var engine="";try{engine=(rep.testEngine&&rep.testEngine.name||"")+" "+((rep.testEngine&&rep.testEngine.version)||"");}catch(e){}return {ran:"true",violation_count:String(v.length),critical_count:String(crit),serious_count:String(ser),moderate_count:String(mod),minor_count:String(min),affected_nodes:String(nodes),violation_ids:ids.join(", "),top_violations:tops.slice(0,15).join(" || ").substring(0,4000),wcag_tags:Object.keys(tags).join(", "),passes_count:String(p.length),incomplete_count:String(inc.length),axe_score:String(score),axe_grade:grade,axe_verdict:verdict,axe_url:String(rep.url||it.page_url||""),axe_engine_version:engine.trim()};)

Nested list fields: report (attribute), page_url (page_url), ran (index), violation_count (index), critical_count (index), serious_count (index), moderate_count (index), minor_count (index), affected_nodes (index), violation_ids (index), top_violations (index), wcag_tags (index), passes_count (index), incomplete_count (index), axe_score (index), axe_grade (index), axe_verdict (index), axe_url (index), axe_engine_version (index)

DOM_Accessibility list
Field details

Selector: css=[id]

Transformations:

  1. script (code: var rows=Array.isArray(value)?value:[];var seen={},dupes=[],invalid=0;for(var i=0;i<rows.length;i++){var id=String(rows[i].idv||"").trim();if(!id)continue;if(!/^[A-Za-z][A-Za-z0-9_\-:\.]*$/.test(id))invalid++;if(seen[id])dupes.push(id);else seen[id]=1;}var dupeUnique={};for(var j=0;j<dupes.length;j++)dupeUnique[dupes[j]]=1;var dk=Object.keys(dupeUnique);return {total_ids:String(rows.length),unique_ids:String(Object.keys(seen).length),duplicate_id_count:String(dupes.length),duplicate_ids:dk.slice(0,40).join(", "),invalid_ids:String(invalid),duplicate_id_verdict:dupes.length===0?"OK":dupes.length+" duplicate id occurrence(s) across "+dk.length+" id(s)"};)

Nested list fields: idv (attribute), total_ids (index), unique_ids (index), duplicate_id_count (index), duplicate_ids (index), invalid_ids (index), duplicate_id_verdict (index)

Form_Accessibility list
Field details

Selector: css=self

Transformations:

  1. script (code: var it=(Array.isArray(value)&&value.length)?value[0]:{};function n(x){var v=parseInt(x,10);return isFinite(v)?v:0;}var inputs=n(it.inputs),noId=n(it.inputs_no_id),labels=n(it.labels),labelsFor=n(it.labels_with_for);var forms=n(it.forms),formsNoAction=n(it.forms_no_action),submits=n(it.submit_buttons);var ifrNoTitle=n(it.iframes_no_title),emptyBtns=n(it.buttons_empty),skip=n(it.skip_links),noLang=n(it.lang_missing);var coverage=inputs?Math.round(((inputs-noId)/inputs)*100):100;var issues=[];if(noLang>0)issues.push("html missing lang attribute");if(noId>0)issues.push(noId+" form control(s) without id/aria-label/title");if(ifrNoTitle>0)issues.push(ifrNoTitle+" iframe(s) without title");if(emptyBtns>0)issues.push(emptyBtns+" empty button(s)");if(!skip&&forms>0)issues.push("No skip link");return {total_form_controls:String(inputs),controls_missing_label:String(noId),label_coverage:String(coverage),forms_total:String(forms),forms_missing_action:String(formsNoAction),submit_buttons_total:String(submits),iframes_missing_title:String(ifrNoTitle),empty_buttons:String(emptyBtns),has_skip_link:String(skip>0),html_lang_missing:String(noLang>0),form_verdict:issues.length?issues.join("; "):"OK"};)

Nested list fields: inputs (count), inputs_no_id (count), labels (count), labels_with_for (count), forms (count), forms_no_action (count), submit_buttons (count), aria_labelled (count), iframes_no_title (count), buttons_empty (count), skip_links (count), lang_missing (count), total_form_controls (index), controls_missing_label (index), label_coverage (index), forms_total (index), forms_missing_action (index), submit_buttons_total (index), iframes_missing_title (index), empty_buttons (index), has_skip_link (index), html_lang_missing (index), form_verdict (index)

Critical_Issues

Item selector: css=html - Required: URL

Fields of dataset "Critical_Issues"
FieldProperty typeSelector / notesDefault
URL page_url
Status_Code variable=last_http_status text
Title text
Field details

Selector: css=head title

Transformations:

  1. trim

Title_Count css=head title count
Description_Count css=meta[name='description'] count
H1_Count css=h1 count
Canonical_Count css=link[rel='canonical'] count
Meta_Robots css=meta[name='robots'] text
X_Robots_Tag variable=last_x_robots_tag text
Viewport_Count css=meta[name='viewport'] count
Img_Total css=img count
Img_Missing_Alt css=img:not([alt]) count
JsonLD_Count css=script[type='application/ld+json'] count
DOM_Node_Count css=* count

Healthy_Pages

Item selector: css=html - Required: URL

Fields of dataset "Healthy_Pages"
FieldProperty typeSelector / notesDefault
URL page_url
Status_Code variable=last_http_status text
Title text
Field details

Selector: css=head title

Transformations:

  1. trim

Title_Chars text
Field details

Selector: css=head title

Transformations:

  1. script (code: return String(String(value==null?'':value).trim().length);, default_value: 0)

H1_Count css=h1 count
Canonical_Count css=link[rel='canonical'] count
Response_Time_ms variable=last_response_time_ms text

Sample output

Sample rows produced by this bot
Page_URLStatus_CodeStatus_ClassIs_SuccessResponse_Time_msHTTP_ProtocolMeta_RobotsCanonical_URLRedirect_Count
https://example.com/ 200 2xx Success true 412 h2 index, follow https://example.com/ 0

File outputs

This bot writes no files.

Delivery destinations

Where finished results go once the run completes.

Results are stored locally; no external delivery is configured.

Export formats: csv.

View the full project configuration on the Config tab.

Frequently asked questions

How do I audit a list of URLs for on-page SEO?
Paste your URLs, one per line, into the required urls_to_audit project setting before launching. The bot works through the list with up to four concurrent workers, and every dataset records a Page_URL column so rows stay attributable to the page they came from. Results are de-duplicated per URL.
What checks does the auditor run on each page?
Nine datasets per page: HTTP response and indexability (status, redirects, robots directives, canonicals, charset, viewport, cookies, security headers with a letter grade); on-page elements (title and meta description with pixel-width truncation checks, heading structure outline, hreflang, Open Graph and Twitter card scoring); content readability (word counts, six readability formulas, text-to-HTML ratio, keyword-density flag, placeholder-text detection); a full links census (internal versus external counts, rel values, anchor-text quality, insecure targets); media and performance (image alt coverage, missing dimensions as a layout-shift risk, modern-format coverage, render-blocking scripts and stylesheets, DOM size and depth, TTFB); structured data (JSON-LD validity, schema types, rich-result eligibility, Microdata and RDFa census); an AXE accessibility audit with severity counts and a grade; and two triage views, Critical_Issues and Healthy_Pages.
Does it check canonical tags and robots directives?
Yes, in depth. The Response_Indexability dataset records the meta robots content, the X-Robots-Tag response header, every robots directive parsed out (noindex, nofollow, noarchive, nosnippet, max-snippet and friends), the source of any noindex signal, canonical count, whether the canonical is self-referencing and absolute, where it resolves to, and a canonical verdict listing concrete issues such as missing, multiple, relative, cross-host or pointing elsewhere. A combined indexability verdict names every blocker found, including meta refresh and non-HTML content types.
How does it score readability?
The Content_Readability dataset computes Flesch Reading Ease, Flesch-Kincaid grade, Gunning Fog, SMOG, Automated Readability and Coleman-Liau from the rendered body text, plus average sentence length, syllables per word, unique-word ratio and estimated reading time. It flags pages as thin when the word count falls below the low_content_words setting (300 by default), classifies length from Empty to Long-form, reports the top non-stopword terms with a keyword-stuffing density verdict, and detects leftover lorem ipsum or placeholder text.
Does it run an accessibility audit?
Yes. The audit_accessibility step runs the AXE engine over the whole document and stores violations, passes and incomplete checks; the Accessibility_AXE dataset then reports counts by severity (critical, serious, moderate, minor), affected nodes, the specific violation ids, WCAG and best-practice tags, a pass-rate score and a letter grade. The step is wrapped in error handling: if the engine cannot run because of a content security policy, a blocking rule or an unreachable engine source, the output says so explicitly instead of pretending the page passed.
What lands in the critical issues file?
Critical_Issues is a filtered triage view containing only pages that fail at least one hard check: HTTP status 400 or above, a title count other than one, an H1 count other than one, more than one meta description, more than one canonical, a missing viewport tag, noindex present in meta robots or the X-Robots-Tag header, more than 25 percent of images missing alt text, or a DOM larger than 1500 nodes. An empty Critical_Issues file after a run is the good-news outcome.
How many URLs can one run audit?
There is no built-in cap on the list length; the practical bounds are your patience and the target server's tolerance. The bot audits with up to four concurrent workers, and each page navigation allows 60 seconds before it is caught, logged and skipped. For large lists, audit in batches and keep an eye on rate-limit responses in the Status_Code column.
Why does the bot block images and stylesheets?
Speed and bandwidth. The configuration blocks image, media, font, stylesheet, websocket, manifest, texttrack, eventsource and other network requests, because the audit reads the DOM, response headers and HTTP metadata rather than pixels. Image tags and their attributes are still fully analysed for alt text, dimensions, lazy loading and formats - only the binary downloads are skipped. The trade-off is that the bot cannot judge visual rendering, so screenshot-dependent checks are out of scope.
Does it crawl my whole site automatically?
No. The bot audits exactly the URLs you submit in the urls_to_audit setting and nothing more; it does not discover pages, follow internal links or parse sitemaps. That is deliberate: an audit list you control is predictable, polite to the server and defensible. If you need discovery, generate the URL list from your own sitemap or analytics export first, then feed it in.
Can it detect structured data and rich result eligibility?
Yes. The Structured_Data dataset counts and parses every JSON-LD block, records parse errors verbatim, walks nested nodes and graphs, and lists all schema types found with counts. It sets flags for Organization, LocalBusiness, WebSite, WebPage, BreadcrumbList, Article, Product, FAQ, HowTo, Review, AggregateRating, VideoObject, Event, Person and Speakable, and produces a rich_result_eligible summary naming the enhancements the page qualifies for. A separate Microdata_RDFa census reports itemtype, itemprop, typeof and property usage so mixed-format markup is visible.
How are duplicate pages detected?
The Content_Readability dataset fingerprints each page's normalised body text with a full content hash and a 64-bit simhash built from three-word shingles, and exposes an eight-character near_duplicate_key. After a run, sort or group that column across your URL list: pages sharing a near-duplicate key have substantially similar body text, which is how you find template duplicates, boilerplate-heavy pages or canonicalisation candidates without eyeballing every file.
Does the bot need an internet connection to run?
Yes on first run: the launcher fetches its configuration by id from the backend at launch, so an internet connection is required at least the first time it starts.
Can I rename the downloaded file?
No. The downloaded file must keep its exact name - the filename carries the per-bot configuration id, and renaming it breaks the configuration fetch at launch.