Google Webmaster Manual Action Penalties

As a webmaster or SEO, there's nothing worse than getting a message from Google Webmaster Tools about a manual action that has been placed on your website. Manual actions are Google's way of demoting or removing web pages or websites as a whole based. They aren't related to Google algorithm changes like Penguin, Panda, Hummingbird, or the others. They are simply Google manually punishing websites for spammy behavior.

How often do manual actions occur?

Here's Google's chart showing the number of manual actions they have imposed over the course of one month.

Manual actions chart

Should you worry about a manual action?

The answer depends on an important factor: whether the manual action affects your website's organic search traffic and rankings.

Some manual actions may not significantly impact your website's organic search traffic and rankings as a whole. They may only impact pages that you no longer care about or pages that do not generate revenue for your business. Instead of fighting the manual action, it may be worth just deleting the page(s) in question.

So how can you tell if a manual action is hurting your website and your business? Start with your Google Analytics. Note the date Google applied the manual action on your website and look at your organic search traffic before and after the action. Does it change significantly?

Organic search traffic

You can also use the compare to previous period option in the date selector to see how your traffic has changed over a longer span of time.

Previous period

If you measure goal conversions in Google Analytics, the traffic may not be as big of a concern as a change in the number of conversions.

Conversions

If you don't see a significant, negative change in organic search traffic and conversions after the manual action was taken on your website, then you may not need to worry about it. Or at least, you don't need to be aggressive about fixing it. On the other hand, if you do see a significant, negative change that is hurting your business, then you will want to be aggressive about fixing it.

Now, let's look at the types of manual actions Google imposes on websites and how to work towards fixing them.

Unnatural Links to Your Site

There are two types of manual actions regarding unnatural links to your website. The first type is where Google acknowledges that the unnatural links they found are out of your control, and therefore they don't take it out on your site's overall rankings. They do suggest you try removing links that you can control, but they don't punish you for the ones you can't.

Unnatural links message

The second type is where Google believes that you have been involved in link schemes and deceptive or manipulative link practices. This one likely affects your site's rankings overall in search.

If your website has received the latter manual action, you may have a long, hard road towards getting it rectified. You will have to show Google that you have made a strong effort to remove as many unnatural links to your website as possible and explain any links that you were unable to remove.

Link Detox can make the process of rebounding from an unnatural links manual action a little less painful. For starters, you can use it to analyze your backlink profile, including the backlinks you export from Google Webmaster Tools.

Link detox high risk links

Link Detox will quickly identify the links that are at the highest risk of being considered unnatural by Google. You can then use LinkResearchTool's integration with Pitchbox to contact webmasters with customized templates and ask them to remove links to your website

Pitchbox integration

With automated follow-ups, you don't have to worry about trying to remember to email people again and again. You can just create a template that does the job for you!

Once you have removed (or attempted to remove) your high-risk links, you can ask Google for a reconsideration request. If they deem you have done a good enough job of removing your unnatural links, they may remove the manual action. You should then monitor your analytics to see if the removal leads to regaining your site's organic traffic and rankings.

If your reconsideration request is denied, then you will need to continue to remove links that are considered at a high or above average risk.

Note that, in either case, no matter how good of a job you do of removing unnatural links, you may not receive a complete recovery of your organic search traffic. Since the links were helping your website rank for specific keywords, having those links removed alone will lower your ranking. You will, therefore, have to work towards link building the Google-approved way.

Unnatural Links from Your Site

Google doesn't only punish websites with unnatural incoming links. They also will impose a manual action on websites with unnatural outbound links. This likely occurs on websites that Google believes is selling links to other websites directly, or by offering dofollow links for sponsored or paid reviews. It can also affect outbound links that are a part of link exchanges or other link schemes.

Manual-Actions-unnatural-links-from-website

If you receive this manual action, your job will be to remove paid links, exchanged links, and other links you have given to other websites.

Alternatively, you can mark them as nofollow. Depending on how many links you have given on a paid basis, you may have a large task at hand.

If none of the outbound links you have on your website are part of a link scheme, then you may want to look for dofollow links in comments on your blog or posts within your forum. You may also want to look for links that are completely unrelated to your website, as those may be the ones Google has identified as unnatural outbound links.

Hacked Site

While a website hacking is not your fault, Google will apply a manual action to your website as soon as they detect malicious code.

The fix for a hacked site is to get all of the malicious code and malware removed as quickly as possible. Once you have done this, the manual action will be removed, and Google will no longer warn visitors to your website that your website is infected.

If you have no idea how to fix your website in the event of a hacking, you can turn to services like Sucuri or SiteLock that specialize in malware monitoring and cleanup. You can prevent these events from happening by paying similar services to constantly monitor and secure your website from hackers and malware.

You can also use Link Alerts to constantly monitor your backlinks since  having too many links from websites affected by malware, may get you in trouble as well.

hacked-website-manual-action

Thin Content

When Google decides that you have content that provides little value, they may impose a manual action for thin content.

Google defines thin content like the following.

Automatically generated content - If a human is not creating your content, then it will likely fall under this category and be considered thin content.

Thin affiliate pages - If the only content your website has is for the purpose of promoting products or services as an affiliate and provides no additional value, it might be considered thin content.

Content from other sources - If you rely upon content scrapers to steal content from other websites, or you get low-quality content from outside contributors (guest posts), then it might be considered thin content.

google-manual-action-thin-content

Doorway pages - If you have multiple pages or websites that you are trying to rank for specific queries that take the user to essentially the same piece of content, then they might be considered doorway pages.

If your website contains any of these types of content, you should look to create new, valuable, and unique content to replace the poor quality and automated content or remove the pages altogether. Once you have updated your website's content, you can submit a reconsideration request.

Not sure about what constitutes high-quality content? Be sure to review Google's guidelines. You can also look at Google's guidelines for content reviewers see what they look for when evaluating web pages for quality.

For those who do not have the time to create the replacement, high-quality content, you should consider outsourcing the content development. You don't want to get in trouble for the same low-quality issues. Therefore you will want to find content creators who specialize in creating high-quality content within your niche or industry.

Pure Spam

The manual action for pure spam can cover a lot of different abuses of the Google Webmaster Guidelines. These include scraped content, automated gibberish, cloaking, and other items that are covered by previously covered manual actions. This includes spammy incoming and outgoing links.

google-manual-action-pure-spam

The only way to recover from this manual action is to clean up any pages and links that are considered to be spam by Google. Depending on the type of spam and how much you have on your website, this may involve completely restructuring your website's architecture, content, on-site optimization, and off-site optimization.

User-Generated Spam

If you own a blog, forum, social network, or membership site with public profiles, you may receive the manual action for user-generated spam based on the behavior of visitors and members of your website. User-generated spam can include blog comments, forum posts, and profiles that are spammy in nature.

manual-actions-user-generated-spam

Depending on the size of your website, number of users and the amount of user-generated content, you may have a tough road ahead of you to fix these issues. You can start by looking through the names of people who have signed up to your website to see if they are real names or computer generated usernames. Google also suggests that you use searches like site:domain.com keyword to find profiles and user-generated content that includes keywords used by spammers related to adult content, online pharmaceuticals, insurance, payday loans, casinos, and similar niches.

Another way to help combat this issue is to implement a system of user voting. This way, real users of your website can down vote content and profiles that they consider spammy or offensive. Instead of you having to find all of the problems on your own, you can let your users help you moderate the community.

If that is not an option and you don't have time to police your community, you may want to hire someone who can moderate the spam out of your website and keep it moderated going forward.

Once you have managed to remove the spammy user-generated content, you can submit a reconsideration request to Google showing that you have cleaned up your website and put assurances in place to make sure the user-generated spam does not build up again.

Cloaking and Sneaky Redirects

redirects

This manual action covers two scenarios. If your website has content that is shown to Google, but not shown to visitors, Google may consider it as cloaking.

If your website has any pages that are indexed in Google, but redirect users to pages that they would not have gone to intentionally, you may have what Google considers sneaky redirects. Sneaky redirects can also apply to redirects that are conditional, such as redirects that are only applied to visitors from Google search.

Sometimes, cloaking and sneaky redirects can be the result of hacking. For example, you see a normal web page as a user, but the code behind the page has been stuffed with various spammy keyword phrases.

If you do not know of any instances of cloaking and sneaky redirects that you have set up on your website, you may want to have your website checked by a hacking or malware removal service to ensure that your website hasn't been hacked in a way that you cannot detect.

Once you have removed instances of cloaking and sneaky redirects, or have determined that both were due to hacking or malware, you can contact Google for your reconsideration request.

Hidden Text and Keyword Stuffing

Manual actions for hidden text and keyword stuffing happen when Google discovers keywords on a page that are not shown to users or an overuse of keywords in the website's optimization. This manual action can sometimes include websites that have been hacked or infected by malware, where the hack injects keywords that you do not know about in your websites code.

If you do not know of any instances of hidden text or overused keywords that you have done to your website, you may want to have your website checked by a hacking or malware removal service to ensure that your website hasn't been hacked in a way that you cannot detect.

Once you have removed instances hidden text or overused keywords, or have determined that both were due to hacking or malware, you can contact Google for your reconsideration request.

Spammy Freehosts

For the most part, Google can tell when a website has spam versus when a website on the same hosting server has spam. But in some cases, if your website is hosted on a server is full of spammy websites, your website might also be lumped into the same grouping.

If you suspect that your website is hosted on a server with other spammy websites, your options are as follows.

You can contact the web host and have them see if they can remove the other spammy websites based on a breach of the host's terms and conditions.

You can ask to have your website moved to another server or a dedicated server that would separate your website from the spammy websites.

You can move to a different hosting company where you can get assurance that your website will not be hosted with other spammy websites.

Once you have removed the association between your website and the spammy websites on your server, you can submit a reconsideration request to Google. Your rankings and organic search traffic should recover.

Spammy Structured Markup

Structured markup for websites can help make your website stand out in search results. If you have knowingly abused structured markup by using markup on your web pages that does not match your content, then you could receive a manual action for spammy structured markup.

Start by reviewing Google's rich snippets guidelines to see if you might have accidentally misused structured markup on your website. Remove any instances of misused structured markup, and you should be able to submit a reconsideration request to Google.

In Conclusion

If you have received a manual action from Google, it isn't the end of the world. You may have a long way to go to repair your website and recover your rankings. It can be done if you are honest about what has gone wrong and can invest the time and resources into making things right again.

"Always follow Google Webmaster Guidelines"

Read More

What is Spyware and Methods of Prevention.?

What is Spyware?

Strictly speaking, the term “spyware” could be used to describe all of the tools and technology with which espionage is conducted – including the poison-tipped umbrella. But in the context of digital/information technology, spyware is any software that obtains information about a person or organization – usually without their knowledge or consent.

At the more legitimate end of the spectrum, this definition would include cookies deposited in your web browser cache when you visit a website, or software used to gather data on you and your habits for marketing and targeting advertising purposes – so long as these operate without your knowing about them.

Spybots or tracking software are terms often used to describe the class of spyware which is installed without a user’s consent – typically through deception (an infected website, pop-up window, or other malware trap) or camouflage (with a spyware program bundled as part of an otherwise legitimate download). Once installed, these programs gather information on the infected target and transmit it to third parties, who may use it for any number of purposes.

A Brief History of Spyware

Back in October 1995, the term “spyware” first appeared in a public forum called Usenet – which was a distributed internet discussion where users could post messages in an email-like format. It featured in an article analyzing the business model then adopted by Microsoft, in a largely conceptual context.

Spyware made its first significant public impact in 1999, when the popular freeware game Elf Bowling was discovered to be laced with tracking software. Also in that year, Steve Gibson of Gibson Research uncovered a form of spyware which, under the pretext of advertising, was actually pilfering confidential information – the first significant intrusion of adware. In response, Gibson went on to develop OptOut, the first anti-spyware program.

A press release for a personal firewall package which appeared in the year 2000 contained specific reference to spyware, and marked its official entry in the modern language of computing.

The Current State of Spyware

As with other forms of malicious application, spyware has enjoyed a rapid evolution since its humble beginnings. In no small part, this may be attributed to the proliferation of websites, portals, file-sharing resources and torrents that enable users to exchange files and bypass officially sanctioned software distributors and app stores to download free software, or cracked and pirated versions of commercial packages.

Authors of spyware have been known to pay the developers of shareware (limited or zero functionality after a certain period, unless you choose to buy) to bundle their tracking software with legitimate packages. They may also re-engineer freeware applications to include their own spyware code.

Beyond the booby-trapped email attachment or infected pop-up window, internet spies may lure unfortunate victims to tainted websites, where spyware is undetectably and automatically downloaded onto a visitor’s machine – the so-called “drive-by download”. This points to the principal weapon that cyber-criminals use in getting their targets to install spyware: stealth.

Spyware works best when the target is unaware of its presence or activities. So today’s spyware practitioners take great pains to ensure that victims have no idea that the software package they’re installing comes bundled with spybots, or that the website they’ve just visited left them with more than what they’ve just seen.

Once established on a host system, spyware may then go on to observe a user’s activities and provide a steady stream of information to its controllers. This could include personal data, contact lists, financial information, account and user credentials, intellectual property, or operational data crucial to businesses. In some cases, the tracking software may also make changes to a system or network to make its own work easier, or as a form of sabotage.

Types of Spyware

There are several forms of tracking software in current circulation, including:

• Adware: The paid advertising which appears on program windows and makes such software notionally free may also mask underlying code which observes a user’s system or activities and relays information to third parties for semi-legitimate marketing or more nefarious purposes. Pornware and riskware may also be used as avenues for tracking software installations.
• Cell Phone or Mobile Spyware: A range of applications, including device-resident software installed by manufacturers to track a user’s position, internet use and activity for targeted advertising and other purposes, and spyware installed manually (by a user, or a malicious outsider gaining access to their phone) or from an infected file download or website.
• Cookies: For the most part, tracer cookies are placed for legitimate purposes – and websites specifically display their cookie and consent policies to the user in their browser window. But they may also be placed surreptitiously, and employed to track users without their knowledge.
• System Monitors: Installed secretly, or slipped in as hidden features of otherwise legitimate-looking security software, spyware system monitors may capture keyboard activity (keyloggers), track and record emails, observe browsing and websites visited, etc.
• Trojans: Seemingly innocent or legitimate software installations and document files that contain spyware in various forms.

Effects of Spyware

The pilfering of confidential personal or corporate data and intellectual property can lead to identity theft, fraud, financial losses, and damage to individual or organizational reputations once the breaches come to light. That’s why spyware is a lucrative option for cyber-criminals, who can sell information on to third parties, or hold people and organizations to ransom over the return of their data or the threat of exposure.

Spyware can also cause real damage to infected systems. There’s often a performance dip associated with a spyware installation, which may manifest as a device or system running slowly, crashes and freezing, increased stress on the processor, higher operating temperatures, battery drains, and so on.

The tracking software may also make changes to system configurations, change port and browser settings, spoof or redirect homepage settings, alter a user’s search engine results, or cause a user’s web browsers to automatically visit infected or fraudulent sites.

Methods of Prevention

New software should be downloaded from approved app stores and manufacturer websites, whenever possible. File-sharing (unless within a secure network environment) and torrent downloads are generally not a good idea. Neither is haphazard clicking on unsolicited email attachments or pop-up windows and advertising.

Security and anti-virus software is always a recommended option – as long as it originates from a reputable manufacturer, and has dedicated anti-spyware facilities. Look for anti-spam filters, cloud-based detection, and virtual encrypted keyboard tools for entering financial information and transactions. Some internet security solutions also offer spyware removal capabilities, in the event of an existing infection.

Device security should also be maintained. So keep a close eye and hand on cell phones and tablets, use password protection and lock screens, and secure all hardware against physical theft or tampering.

Origian from: https://blog.finjan.com/the-past-and-present-state-of-spyware/

Read More

What is Affiliate Marketing?

A common way to make money online is through affiliate marketing.Affiliate marketing is internet advertising that allows any online business to affiliate themselves with web site owners (known as affiliates or publishers) using affiliate programs. Affiliates make money by generating sales, leads and traffic for the Merchants business. Merchants who sell goods and services online or seek other affiliate business activity typically use one of the affiliate program commission payment models below:

Cost Per Sale (CPA) 

A visitor referred by the Affiliate purchases goods and services from the Merchant. This payment structure is referred to as - cost per sale or cost per acquisition (CPA). 

Cost Per Lead (CPL) 

A visitor referred by the Affiliate completes a form on the Merchants web site. This payment structure is referred to as - cost per lead (CPL). 

Cost Per Click (CPC)

A visitor to the Affiliates web site clicks on a Merchant's banner and visits the Merchant's web site. This payment structure is referred to as - cost per click (CPC).

Merchants - Affiliate marketing is a excellent way to build your own large online sales team and drive customers to your affiliate business web site on a 'pay for performance' basis. As a Merchant you can build your own pay per sale, pay per lead or pay per click Affiliate program and use graphic, text and custom HTML/Flash advertisements to promote and drive customers to your web site. Affiliates will join your program and display your advertisements on their web sites, sending customers and traffic back to your site. All the Affiliate programs you build are free and provide you with comprehensive online management and sale reporting tools, so that you can track your sales, impressions and the customer traffic reaching your web site. 

You have full control over your Affiliate program, what country your traffic comes from and which Affiliates promote your program. You also have complete control over the rate paid for each sale(cpa), lead(cpl), click(cpc) or the display of one thousand of your banners(cpm). You may also choose to use our additional promotional service ensuring that your program reaches a very wide and large US, European, Asian or Australian based customer audience. 

Affiliates - Affiliate marketing allows you to earn high income from your web site while providing related services to your visitors.You can start earning money today with your web site by joining one of the thousands of Affiliate programs located on this site. Many online businesses offer Affiliate programs as a way of generating sales and traffic for their online businesses. These companies will pay you high commissions based on the traffic they receive from advertisements you place on your site. You can choose from pay-per-sale, pay-per-lead, pay-per-click or pay-per-display Affiliate programs. 

All the listed programs are free to join and provide you with online statistics so that you can track your commissions. Once you have joined an Affiliate program, you will be paid according to the programs pay type. (i.e.) a pay-per-sale program pays you each time a sale is made by a customer sent from your web site; a pay-per-lead programs pays you each time you forward a lead to the Merchant; a pay-per-click program pays each time an advertisement is clicked and pay-per-display of 1000 of a merchants banners(cpm).

Following are some of the common terms associated with affiliate marketing:

Affiliates: Publishers like you and me who are using affiliate program links to promote and make sales.

Affiliate marketplace: There are many marketplaces like Shareasale, CJ and Clickbank, which work as central databases for affiliate programs in different niches.

Affiliate software: Software used by companies to create an affiliate program for their product, for example: iDevaffiliate.

Affiliate link: Special tracking link offered by your affiliate program to track the progress of your affiliate promotion.

Affiliate ID: Similar to the affiliate link, but many affiliate programs offer a unique ID which you can add to any page of the product site.

Payment mode: Different affiliate programs offer different methods of payment. For example: check, wire transfer, Paypal and many more.

Affiliate Manager/OPM: Many companies have dedicated affiliate managers to help publishers to earn more by giving them optimization tips.

Commission percentage/amount: The amount or percentage you will be receiving in affiliate income from every sale.

2-tier affiliate marketing: This is a great way of making money from an affiliate program. With this method you recommend that others join affiliate programs, and you receive a commission when a sub-affiliate makes a sale, (similar to MLM or multi-level marketing.) This income is popularly known as sub-affiliate commission.

Landing pages: A unique product sales or demo page used for the purpose of increasing sales. Most of the programs that you will be promoting have many landing pages, and you can run A/B testing to see which pages convert best for you.

Custom affiliate income/ account: Unlike a generic affiliate account, many companies offer custom affiliate income to people making the most affiliate sales for them.

Link clocking: Most of the affiliate tracking links are ugly. Using a link clocking technique like URL shorteners, Thirsty Affiliates, etc., you can turn ugly links into links that can be read and understood by your readers.

Custom coupons: Many programs allow affiliates to create custom coupons which are also used to track sales. Custom discount coupons help you to increase affiliate sales as well.

Who can be an affiliate?

Any person or company who owns a website from which they can send traffic via links (banners or text links) to the advertiser’s website could be an affiliate. In addition to the above, an affiliate can also be anyone who sends users to the advertiser’s website via Pay per Click (PPC) campaigns the affiliate has created, on Google, Facebook and other search engines. The rise in popularity of social media websites like Facebook and Twitter has created another type of affiliate, who sends traffic to advertisers via their social media profiles and pages.

Affiliates send traffic to advertisers in order to receive a pre-agreed sum of money or level of commission, if the users they have sent perform the agreed action (usually a sale). The simplicity of the affiliate model means that anyone can become an affiliate, from a price comparison website, to a content site, a forum or even a blog.

Who can be an advertiser?

Any company owning a website where users can perform some sort of transaction could be an advertiser. Most of the times, this transaction refers to an order/sale, in which case the advertiser has an e-commerce website (e-shop), but it can also refer to other actions like the completion of an order form, sign up to the newsletter, catalogue request etc.

Read More

Best Security Apps to Keep Your iPhone and iPad Secure

Smartphones and tablets are intensely personal gadgets with a lot of important information stored up within them. From photos to contacts and email to card details, people store up a lot of personal, sensitive information on their smartphones. That’s why the market overflows with tons of security apps built specifically for the iPhone and the iPad. 

The security apps for iDevices are varied in nature; some of these help you store passwords securely, while some of them help you track a lost iDevice and a few others help you keep your files and media secure.

"Password Management"

mSecure

When it comes to secure password management with feature like auto-fill, groups, sync etc., there are two bigwigs: 1PasswordLock and mSecure. Personally, mSecure is as good as 1PasswordLock which is more popular. mSecure brings a fantastic level of security to protect and manage your various passwords. Options are simplified but quite sufficient; you can ask the app to generate highly secure passwords, you can segregate your accounts based on the type and management is pretty easy.

Price: $9.99
Download mSecure

"Device Tracking"

Lookout
Lookout works almost the same way as Apple’s very own Find My iPhone. It tracks your device over GPS, lets you locate a lost iPhone via the web using the Lookout.com account, and it can also ring your iPhone remotely via any web browser or send a message to the iPhone if it gets lost. There are a few other features that don’t come with Find My iPhone. For instance, Lookout automatically saves the last-known location of your iPhone before it runs out of battery (or is switched off). And the app will also backup your contacts just in case you need them instantly.

Download Lookout

"Find My iPhone"

Apple’s stock GPS-tracking app is by far the best tracking and security iPhone/iPad app. It’s one of the coolest additions to the iOS firmware. Once installed on your iPhone, you can track your iPhone’s movements, and even remotely control it. Notable options include the one where you can make your iPhone raise an alarm (so whoever has it is taken by surprise) and erase all data before it gets misused (in a process called Remote Wipe).

When an iPhone is lost, you can track it by logging into your account on iCloud.com.

Price: Free
Download Find My iPhone

"File/Content Security"

Best Phone Security Pro

This app will protect your iPhone from unauthorized access: well, it’s basically a passcode lock system just like the stock iOS 7 passcode lock. The difference is that when you’ve got this app running, anyone trying to gain access to your iPhone will be warded off with an alarm sound that you can set. You can actually record a particular alarm sound yourself and use that. But even more interestingly, this app will capture the photo of anyone who enters the wrong passcode in their attempts to get into your iPhone.

Price: $0.99
Download Best Phone Security Pro

"iPhone Used For Surveillance"

Foscam Surveillance Pro

A few cheap IP cameras, this app and you’re the security guy manning the cameras! The app functions as a security surveillance module where you can view the live feed from up to six IP cameras set up in various locations. You can also control the movement if it’s a Foscam IP camera! It’s a cheap and perfect solution for small-office owners.

Price: $4.99
Download Foscam Surveillance Pro

Read More

Learn about Web Crawlers Search engines and User-Agents

Web Crawlers

Web crawlers, also known as web spiders or internet bots, are programs that browse the web in an automated manner for the purpose of indexing content. Crawlers can look at all sorts of data such as content, links on a page, broken links, sitemaps, and HTML code validation.

  

  

Search engines like Google, Bing, and Yahoo use crawlers to properly index downloaded pages so that users can find them them faster and more efficiently when they are searching. Without crawlers there would be nothing to tell them that your website has new and fresh content. Sitemaps also can play a part in that process. So web crawlers, for the most part, are a good thing. However there are also issues sometimes when it comes to scheduling and load as a crawler might be constantly polling your site. And this is where a robots.txt file comes into play. This file can help control the crawl traffic and ensure that it doesn’t overwhelm your server.

Web crawlers identify themselves to a web server by using the User-agent field in an HTTP request, and each crawler has their own unique identifier. Most of the time you will need to examine your web server referrer logs to view web crawler traffic.

Robots.txt
By placing a robots.txt file at the root of your web server you can define rules for web crawlers such as allow or disallow that they must follow. You can apply generic rules which apply to all bots or get more granular and specify their specific User-agent string.

Learn more about the Top Search engine Bots
There are hundreds of web crawlers and bots scouring the internet but below is a list of popular web crawlers and bots that we have  been collected based on ones that we see on a regular basis within our web server logs.

Googlebot: Googlebot is Google’s web crawling bot (sometimes also called a “spider”). Googlebot uses an algorithmic process: computer programs determine which sites to crawl, how often, and how many pages to fetch from each site. Googlebot’s crawl process  begins with a list of webpage URLs, generated from previous crawl processes and augmented with Sitemap data provided by webmasters. As Googlebot visits each of these websites it detects links (SRC and HREF) on each page and adds them to its list of pages to crawl. New sites, changes to existing sites, and dead links are noted and used to update the Google index.

"Google+"Another one you might see popup is Google+. When a user shares a URL on Google+ or an app writes an app activity, Google+ attempts to fetch the content and create a snippet to provide a summary of the linked content. This service is different than the Googlebot that crawls and indexes your site. These requests do not honor robots.txt or other crawl mechanisms because this is a user-initiated request.

"Most Popular And Best Internet Search Engines in the World"

"Baiduspider " Baiduspider is a robot of Baidu Chinese search engine. Baidu (Chinese: 百度; pinyin: Bǎidù) is the leading Chinese search engine for websites, audio files, and images.

"MSN Bot/Bingbot" This is a web-crawling robot (type of Internet bot), deployed by Microsoft to supply Bing (search engine). It collects documents from the web to build a searchable index for the Bing (search engine).

"Slurp Bot"Yahoo Search results come from the Yahoo web crawler Slurp and Bing’s web crawler, as a lot of Yahoo is now powered by Bing. Sites should allow Yahoo Slurp access in order to appear in Yahoo Mobile Search results.dditionally, Slurp does the following:

Collects content from partner sites for inclusion within sites like Yahoo News, Yahoo Finance and Yahoo Sports.
Accesses pages from sites across the Web to confirm accuracy and improve Yahoo’s personalized content for our users.

"Yandex Bot" Yandex bot is Yandex’s search engine’s crawler. Yandex is a Russian Internet company which operates the largest search engine in Russia with about 60% market share in that country. Yandex ranked as the fifth largest search engine worldwide with more than 150 million searches per day as of April 2012 and more than 25.5 million visitors.

"Soso Spider " Soso.com is a Chinese search engine owned by Tencent Holdings Limited, which is well known for its other creation QQ. Soso.com is ranked as the 36th most visited website in the world and the 13th most visited website in China, according to Alexa Internet. On an average, Soso.com gets 21,064,490 page views everyday.


"DuckDuckBot"DuckDuckBot is the Web crawler for DuckDuckGo, a search engine that has become quite popular lately as it is known for privacy and not tracking you. It now handles over 12 million queries per day. DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (their crawler) and crowd-sourced sites (Wikipedia). They also have more traditional links in the search results, which they source from Yahoo!, Yandex and Bing.

"Baiduspider"Baiduspider is the official name of the Chinese Baidu search engine’s web crawling spider. It crawls web pages and returns updates to the Baidu index. Baidu is the leading Chinese search engine that takes an 80% share of the overall search engine market of China Mainland.

"Sogou Spider"Sogou Spider is the web crawler for Sogou.com, a leading Chinese search engine that was launched in 2004. it has a rank of 103 in Alexa’s internet rankings. Note: The Sogou web spider does not respect the robots.txt internet standard, and is therefore banned from many web sites because of excessive crawling.

"Exabot"Exabot is a web crawler for Exalead, which is a search engine based out of France. It was founded in 2000 and now has more than 16 billion pages currently indexed.

"Facebook External Hit"Facebook allows its users to send links to interesting web content to other Facebook users. Part of how this works on the Facebook system involves the temporary display of certain images or details related to the web content, such as the title of the webpage or the embed tag of a video.

"Alexa Crawler"Ia_archiver is the web crawler for Amazon’s Alexa internet rankings. As you probably know they collect information to show rankings for both local and international sites.

"Google Feedfetcher" Used by Google to grab RSS or Atom feeds when users choose to add them to their Google homepage or Google Reader. Feedfetcher collects and periodically refreshes these user-initiated feeds, but does not index them in Blog Search or Google’s other search services (feeds appear in the search results only if they’ve been crawled by Googlebot).

 

 

http://www.bing.com/webmaster/ping.aspx?siteMap=http://www.undocopy.com/sitemap.xml

www.bing.com/webmaster/ping.aspx?siteMap=http://www.undocopy.com/sitemap.xml

http://www.google.com/webmasters/sitemaps/ping?sitemap=http://www.undocopy.com/sitemap.xml
     

Read More

What is Ransomware and Easy Steps To Keep Your data and System Protected

Ransomware is malware for data kidnapping, an exploit in which the attacker encrypts the victim's data and demands payment for the decryption key. 

Ransomware spreads through e-mail attachments, infected programs and compromised websites. A ransomware malware program may also be called a cryptovirus, cryptotrojan or cryptoworm.Attackers may use one of several different approaches to extort money from their victims:

After a victim discovers he cannot open a file, he receives an email ransom note demanding a relatively small amount of money in exchange for a private key. The attacker warns that if the ransom is not paid by a certain date, the private key will be destroyed and the data will be lost forever.

The victim is duped into believing he is the subject of an police inquiry. After being informed that unlicensed software or illegal web content has been found on his computer, the victim is given instructions for how to pay an electronic fine.

Must Read : Bitcoin use in Ransomware 

The malware surreptitiously encrypts the victim's data but does nothing else. In this approach, the data kidnapper anticipates that the victim will look on the Internet for how to fix the problem and makes money by selling anti-ransomware software on legitimate websites.To protect against data kidnapping, experts urge that users backup data on a regular basis. If an attack occurs, do not pay a ransom. Instead, wipe the disk drive clean and restore data from the backup.

Ransomware is a sophisticated piece of malware that blocks the victim’s access to his/her files.

"There are two types of ransomware in circulation:"

"Encrypting ransomware:" which incorporates advanced encryption algorithms. It’s designed to block system files and demand payment to provide the victim with the key that can decrypt the blocked content. Examples include CryptoLocker, Locky, CrytpoWall and more.

"Locker ransomware:" which locks the victim out of the operating system, making it impossible to access the desktop and any apps or files. The files are not encrypted in this case, but the attackers still ask for a ransom to unlock the infected computer. Examples include the police-themed ransomware or Winlocker.

"Ransomware has some key characteristics that set it apart from other malware:"

It features unbreakable encryption, which means that you can’t decrypt the files on your own (there are various decryption tools released by cyber security researchers – more on that later);

It has the ability to encrypt all kinds of files, from documents to pictures, videos, audio files and other things you may have on your PC;

It can scramble your file names, so you can’t know which data was affected. This is one of the social engineering tricks used to confuse and coerce victims into paying the ransom;

It will add a different extension to your files, to sometimes signal a specific type of ransomware strain;

It will display an image or a message that lets you know your data has been encrypted and that you have to pay a specific sum of money to get it back;

It requests payment in Bitcoins, because this crypto-currency cannot be tracked by cyber security researchers or law enforcements agencies;

Usually, the ransom payments has a time-limit, to add another level of psychological constraint to this extortion scheme. Going over the deadline typically means that the ransom will increase, but it can also mean that the data will be destroyed and lost forever.

It uses a complex set of evasion techniques to go undetected by traditional antivirus (more on this in the “Why ransomware often goes undetected by antivirus” section);

It often recruits the infected PCs into botnets, so cyber criminals can expand their infrastructure and fuel future attacks;

It can spread to other PCs connected in a local network, creating further damage;

It frequently features data exfiltration capabilities, which means that ransomware can extract data from the affected computer (usernames, passwords, email addresses, etc.) and send it to a server controlled by cyber criminals;

It sometimes includes geographical targeting, meaning the ransom note is translated into the victim’s language, to increase the chances for the ransom to be paid.

"Where does the current wave of ransomware infection come from?"

Even though most companies have extensive security mechanisms in place, such as virus scanners, firewalls, IPS systems, anti-SPAM/anti-virus-email-gateways and web filters, we are currently witnessing large numbers of infections worldwide with ransomware infections, such as Cryptowall, TeslaCrypt and Locky. Files on computers and network drives are encrypted as part of these infections in order to blackmail the users of these computers to pay a sum of money, usually in the region of USD 200-500, for the decryption tool.

"A common infection scenario may look like this:"

A user receives an email that comes from a seemingly plausible sender with an attached document, a parcel service with attached delivery information or anexternal company with an attached invoice.

The email attachment contains an MS Word or Excel document with an embeddedmacro. If the recipient opens the document a macro will attempt to startautomatically, executing the following actions:

It tries to download the actual ransomware payload from a series of webaddresses that only exist momentarily. If a web address cannot be reached, thenext one is accessed until the payload has been downloaded successfully.

"The macro executes the ransomware"

The ransomware contacts the command & control server of the attacker,sends information about the infected computer and downloads an individual public key for this computer.

Files of certain types (Office documents, database files, PDFs, CAD documents,HTML, XML etc.) are then encrypted on the local computer and on all accessible network drives with this public key.

Automatic backups of the Windows operating system (shadow copies) are often deleted to prevent this type of data recovery.

"Best practices to apply immediately"

Backup regularly and keep a recent backup copy off-site. There are dozens of ways other than ransomware that files can suddenly vanish, such as fire, flood, theft, a dropped laptop or even an accidental delete. Encrypt your backup and you won’t have to worry about the backup device falling into the wrong hands.

Don’t enable macros in document attachments received via email. Microsoft deliberately turned off auto-execution of macros by default many years ago as a security measure. A lot of malware infections rely on persuading you to turn macros back on, so don’t do it!

Be cautious about unsolicited attachments.
The crooks are relying on the dilemma that you shouldn’t open a document until you are sure it’s one you want, but you can’t tell if it’s one you want until you open it. If in doubt, leave it out.

Don’t give yourself more login power than you need. Most importantly, don’t stay logged in as an administrator any longer than is strictly necessary, and avoid browsing, opening documents or other “regular work” activities while you haveadministrator rights.

Consider installing the Microsoft Office viewers. 

These viewer applications let you see what documents look like without opening them in Word or Excel itself. Inparticular, the viewer software doesn’t support macros at all, so you can’t enablemacros by mistake!

Patch early, patch often. 

Malware that doesn’t come in via document macros often relies on security bugs in popular applications, including Office, your browser, Flash and more. 

The sooner you patch, the fewer open holes remain forthe crooks to exploit. Keep informed about new security features added to your business applications. Forexample, Office 2016 now includes a control called "Block macros from running inOffice files from the internet" which helps protect you from external malicious content without stopping you using macros internally.

Open .JS files with Notepad by default.

This helps protect against JavaScript borne malware by enabling you to identify the file type and spot suspicious files.

Show files with their extensions. 

Malware authors increasingly try to disguise the actual file extension to trick you into opening them. Avoid this by displaying files with their extensions at all times.

Read More

What is Bitcoin and How does Bitcoin work?

Bitcoin is a form of digital currency, created and held electronically. No one controls it. Bitcoins aren’t printed, like dollars or euros – they’re produced by people, and increasingly businesses, running computers all around the world, using software that solves mathematical problems.

Bitcoin is a type of digital asset, most commonly defined as a virtual, digital or crypto- currency.

Bitcoin was invented by Satoshi Nakamoto, who published the payment system in a whitepaper in 2008. Released as open-source software in 2009, the bitcoin system is peer-to-peer which means that transactions take place without the need for a third-party. These transactions or payments are then recorded in a public distributed ledger called the blockchain, which uses bitcoin as its unit of account.

The blockchain is the backbone of bitcoin, serving as a financial ledger without any trusted central authority - this means that it is decentralized and no single person or institution owns the currency.With every new transaction, recordings are added as 'blocks', which each block added in a linear, chronological order. Described in simple traditional banking terms - the blockchain is the full history of banking transactions, while the blocks are like individual bank statements. Bitcoins do not physically exist, instead there are only these records of transactions and balances.

The unit of currency in the bitcoin system is, simply, bitcoin. There are three symbols used to express bitcoin - BTC, XBT and ?. Bitcoins can be split down into alternative units such as millibitcoin (mBTC), microbitcoin (µBTC) and satoshi. Satoshi, named in honour of the bitcoin founder, is one hundred millionth of a bitcoin - the smallest amount possible.

You can acquire bitcoins as payment for a service or goods or through purchase at bitcoin exchanges.

Bitcoins, however, can be created by anyone - all you have to do is help process payments into the distributed ledger. By offering your own compute power to verify and record payments, bitcoins are created as a reward or payment for your services. This activity is widely known as bitcoin mining.

Bitcoin is an anonymous digital currency.Bitcoin is not real money. It's an online "currency"—virtual tokens that can be exchanged for goods and services at places that accept it, the same way you'd give someone a dollar for a cookie. But unlike a dollar, a Bitcoin has no serial number or any possible mechanism that could be used to trace it back to a buyer or seller. This makes it attractive to drug dealers and/or privacy advocates.

itcoin is a digital currency (also called crypto-currency) that is not backed by any country's central bank or government. Bitcoins can be traded for goods or services with vendors who accept Bitcoins as payment.

Bitcoin-to-Bitcoin transactions are made by digitally exchanging anonymous, heavily encrypted hash codes across a peer-to-peer (P2P) network. The P2P network monitors and verifies the transfer of Bitcoins 

The basics for a new user
As a new user, you can get started with Bitcoin without understanding the technical details. Once you have installed a Bitcoin wallet on your computer or mobile phone, it will generate your first Bitcoin address and you can create more whenever you need one. You can disclose your addresses to your friends so that they can pay you or vice versa. In fact, this is pretty similar to how email works, except that Bitcoin addresses should only be used once.

Balances - block chain

The block chain is a shared public ledger on which the entire Bitcoin network relies. All confirmed transactions are included in the block chain. This way, Bitcoin wallets can calculate their spendable balance and new transactions can be verified to be spending bitcoins that are actually owned by the spender. The integrity and the chronological order of the block chain are enforced with cryptography.

Transactions - private keys

A transaction is a transfer of value between Bitcoin wallets that gets included in the block chain. Bitcoin wallets keep a secret piece of data called a private key or seed, which is used to sign transactions, providing a mathematical proof that they have come from the owner of the wallet. The signature also prevents the transaction from being altered by anybody once it has been issued. All transactions are broadcast between users and usually begin to be confirmed by the network in the following 10 minutes, through a process called mining.

Processing - mining

Mining is a distributed consensus system that is used to confirm waiting transactions by including them in the block chain. It enforces a chronological order in the block chain, protects the neutrality of the network, and allows different computers to agree on the state of the system. To be confirmed, transactions must be packed in a block that fits very strict cryptographic rules that will be verified by the network. These rules prevent previous blocks from being modified because doing so would invalidate all following blocks. Mining also creates the equivalent of a competitive lottery that prevents any individual from easily adding new blocks consecutively in the block chain. This way, no individuals can control what is included in the block chain or replace parts of the block chain to roll back their own spends.

Going down the rabbit hole

This is only a very short and concise summary of the system. If you want to get into the details, you can read the original paper that describes the system's design, read the developer documentation, and explore the Bitcoin wiki.

Read More