Helper class to create VPRecipies from IR instructions. More...

#include "Transforms/Vectorize/VPRecipeBuilder.h"

Public Member Functions
VPValue *	createBlockInMask (BasicBlock *BB, VPlanPtr &Plan)
	A helper function that computes the predicate of the block BB, assuming that the header block of the loop is set to True. More...

VPValue *	createEdgeMask (BasicBlock Src, BasicBlock Dst, VPlanPtr &Plan)
	A helper function that computes the predicate of the edge between SRC and DST. More...

VPInterleaveRecipe *	tryToInterleaveMemory (Instruction *I, VFRange &Range, VPlanPtr &Plan)
	Check if belongs to an Interleave Group within the given VF `Range`,. More...

VPWidenMemoryInstructionRecipe *	tryToWidenMemory (Instruction *I, VFRange &Range, VPlanPtr &Plan)
	Check if is a memory instruction to be widened for `Range.Start` and potentially masked. More...

VPWidenIntOrFpInductionRecipe *	tryToOptimizeInduction (Instruction *I, VFRange &Range)
	Check if an induction recipe should be constructed for within the given VF `Range`. More...

VPBlendRecipe *	tryToBlend (Instruction *I, VPlanPtr &Plan)
	Handle non-loop phi nodes. More...

bool	tryToWiden (Instruction I, VPBasicBlock VPBB, VFRange &Range)
	Check if `I` can be widened within the given VF `Range`. More...

VPRegionBlock *	createReplicateRegion (Instruction I, VPRecipeBase PredRecipe, VPlanPtr &Plan)
	Create a replicating region for instruction `I` that requires predication. More...

	VPRecipeBuilder (Loop OrigLoop, const TargetLibraryInfo TLI, const TargetTransformInfo TTI, LoopVectorizationLegality Legal, LoopVectorizationCostModel &CM, VPBuilder &Builder)

bool	tryToCreateRecipe (Instruction Instr, VFRange &Range, VPlanPtr &Plan, VPBasicBlock VPBB)
	Check if a recipe can be create for `I` withing the given VF `Range`. More...

VPBasicBlock *	handleReplication (Instruction I, VFRange &Range, VPBasicBlock VPBB, DenseMap< Instruction , VPReplicateRecipe > &PredInst2Recipe, VPlanPtr &Plan)
	Build a VPReplicationRecipe for `I` and enclose it within a Region if it is predicated. More...

Detailed Description

Helper class to create VPRecipies from IR instructions.

Definition at line 26 of file VPRecipeBuilder.h.

Constructor & Destructor Documentation

◆ VPRecipeBuilder()

llvm::VPRecipeBuilder::VPRecipeBuilder	(	Loop *	OrigLoop,
		const TargetLibraryInfo *	TLI,
		const TargetTransformInfo *	TTI,
		LoopVectorizationLegality *	Legal,
		LoopVectorizationCostModel &	CM,
		VPBuilder &	Builder
	)

inline

Definition at line 107 of file VPRecipeBuilder.h.

References handleReplication(), and tryToCreateRecipe().

Member Function Documentation

◆ createBlockInMask()

VPValue * VPRecipeBuilder::createBlockInMask	(	BasicBlock *	BB,
		VPlanPtr &	Plan
	)

A helper function that computes the predicate of the block BB, assuming that the header block of the loop is set to True.

It returns the entry mask for the block BB.

Definition at line 6349 of file LoopVectorize.cpp.

References assert(), llvm::InnerLoopVectorizer::Builder, llvm::LoopBase< BlockT, LoopT >::contains(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::LoopVectorizationLegality::getPrimaryInduction(), llvm::VPInstruction::ICmpULE, if(), llvm::InnerLoopVectorizer::Legal, llvm::InnerLoopVectorizer::OrigLoop, and llvm::predecessors().

◆ createEdgeMask()

VPValue * VPRecipeBuilder::createEdgeMask	(	BasicBlock *	Src,
		BasicBlock *	Dst,
		VPlanPtr &	Plan
	)

A helper function that computes the predicate of the edge between SRC and DST.

Definition at line 6318 of file LoopVectorize.cpp.

References assert(), llvm::InnerLoopVectorizer::Builder, llvm::dyn_cast(), llvm::BranchInst::getCondition(), llvm::BranchInst::getSuccessor(), llvm::BasicBlock::getTerminator(), llvm::is_contained(), llvm::BranchInst::isConditional(), and llvm::predecessors().

◆ createReplicateRegion()

VPRegionBlock * VPRecipeBuilder::createReplicateRegion	(	Instruction *	I,
		VPRecipeBase *	PredRecipe,
		VPlanPtr &	Plan
	)

Create a replicating region for instruction I that requires predication.

PredRecipe is a VPReplicateRecipe holding I.

Definition at line 6653 of file LoopVectorize.cpp.

References assert(), llvm::VPBlockUtils::connectBlocks(), llvm::Instruction::getOpcodeName(), llvm::Instruction::getParent(), llvm::Value::getType(), llvm::VPBlockUtils::insertTwoBlocksAfter(), and llvm::Type::isVoidTy().

◆ handleReplication()

VPBasicBlock * VPRecipeBuilder::handleReplication	(	Instruction *	I,
		VFRange &	Range,
		VPBasicBlock *	VPBB,
		DenseMap< Instruction , VPReplicateRecipe > &	PredInst2Recipe,
		VPlanPtr &	Plan
	)

Build a VPReplicationRecipe for I and enclose it within a Region if it is predicated.

Returns: VPBB augmented with this new recipe if I is not predicated, otherwise; a new VPBasicBlock that succeeds the new Region. Update the packing decision of predicated instructions if they feed I. Range.End may be decreased to ensure same recipe behavior from Range.Start to Range.End.

Definition at line 6614 of file LoopVectorize.cpp.

References llvm::VPBasicBlock::appendRecipe(), assert(), llvm::dbgs(), llvm::SmallVectorBase::empty(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::end(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::find(), llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), llvm::VPBlockBase::getSuccessors(), I, llvm::VPBlockUtils::insertBlockAfter(), LLVM_DEBUG, and llvm::User::operands().

Referenced by VPRecipeBuilder().

◆ tryToBlend()

VPBlendRecipe * VPRecipeBuilder::tryToBlend	(	Instruction *	I,
		VPlanPtr &	Plan
	)

Handle non-loop phi nodes.

Currently all such phi nodes are turned into a sequence of select instructions as the vectorizer currently performs full if-conversion.

Definition at line 6485 of file LoopVectorize.cpp.

References assert(), llvm::dyn_cast(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::PHINode::getIncomingBlock(), llvm::PHINode::getNumIncomingValues(), llvm::Instruction::getParent(), I, llvm::tgtok::In, llvm::InnerLoopVectorizer::OrigLoop, and llvm::SmallVectorTemplateBase< T >::push_back().

◆ tryToCreateRecipe()

bool VPRecipeBuilder::tryToCreateRecipe	(	Instruction *	Instr,
		VFRange &	Range,
		VPlanPtr &	Plan,
		VPBasicBlock *	VPBB
	)

Check if a recipe can be create for I withing the given VF Range.

If a recipe can be created, it adds it to VPBB.

Definition at line 6681 of file LoopVectorize.cpp.

References llvm::VPBasicBlock::appendRecipe(), assert(), llvm::LoopBlocksDFS::beginRPO(), llvm::LoopBase< BlockT, LoopT >::blocks(), llvm::MCID::Branch, llvm::InnerLoopVectorizer::Builder, llvm::LoopVectorizationCostModel::CM_Interleave, llvm::dbgs(), DFS(), llvm::VPBlockUtils::disconnectBlocks(), llvm::dyn_cast(), llvm::LoopBase< BlockT, LoopT >::empty(), llvm::VPBasicBlock::empty(), EnableVPlanNativePath, llvm::VFRange::End, llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::end(), llvm::SmallPtrSetImpl< PtrType >::end(), llvm::LoopBlocksDFS::endRPO(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::find(), llvm::SmallPtrSetImpl< PtrType >::find(), llvm::raw_ostream::flush(), llvm::BranchInst::getCondition(), llvm::LoopVectorizationLegality::getInductionVars(), llvm::InterleaveGroup< InstTy >::getInsertPos(), llvm::LoopBase< BlockT, LoopT >::getLoopLatch(), llvm::LoopVectorizationLegality::getPrimaryInduction(), llvm::VPBlockBase::getSingleSuccessor(), llvm::LoopVectorizationLegality::getSinkAfter(), I, llvm::VPBlockUtils::insertBlockAfter(), llvm::BranchInst::isConditional(), llvm::InnerLoopVectorizer::Legal, llvm::InnerLoopVectorizer::LI, LLVM_DEBUG, llvm::make_range(), llvm::InnerLoopVectorizer::OrigLoop, llvm::LoopBlocksDFS::perform(), llvm::VPBlockBase::setName(), llvm::VFRange::Start, llvm::InnerLoopVectorizer::TLI, llvm::InnerLoopVectorizer::TTI, llvm::InnerLoopVectorizer::VF, and llvm::VPlanHCFGTransforms::VPInstructionsToVPRecipes().

Referenced by VPRecipeBuilder().

◆ tryToInterleaveMemory()

VPInterleaveRecipe * VPRecipeBuilder::tryToInterleaveMemory	(	Instruction *	I,
		VFRange &	Range,
		VPlanPtr &	Plan
	)

Check if belongs to an Interleave Group within the given VF Range,.

Returns: true in the first returned value if so and false otherwise. Build a new VPInterleaveGroup Recipe if is the primary member of an IG for Range.Start, and provide it as the second returned value. Note that if is an adjunct member of an IG for Range.Start, the; value is <true, nullptr>, as it is handled by another recipe. Range.End may be decreased to ensure same decision from Range.Start to Range.End.

Definition at line 6390 of file LoopVectorize.cpp.

References assert(), llvm::LoopVectorizationCostModel::CM_Interleave, function, llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), llvm::InterleaveGroup< InstTy >::getInsertPos(), llvm::Instruction::getParent(), I, llvm::LoopVectorizationLegality::isMaskRequired(), llvm::InnerLoopVectorizer::Legal, llvm::BitmaskEnumDetail::Mask(), and llvm::InnerLoopVectorizer::VF.

◆ tryToOptimizeInduction()

VPWidenIntOrFpInductionRecipe * VPRecipeBuilder::tryToOptimizeInduction	(	Instruction *	I,
		VFRange &	Range
	)

Check if an induction recipe should be constructed for within the given VF Range.

If so build and return it. If not, return null. Range.End may be decreased to ensure same decision from Range.Start to Range.End.

Definition at line 6453 of file LoopVectorize.cpp.

References function, llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), llvm::LoopVectorizationLegality::getInductionVars(), llvm::InductionDescriptor::getKind(), llvm::User::getOperand(), llvm::InductionDescriptor::IK_FpInduction, llvm::InductionDescriptor::IK_IntInduction, llvm::InnerLoopVectorizer::Legal, llvm::MapVector< KeyT, ValueT, MapType, VectorType >::lookup(), and llvm::InnerLoopVectorizer::VF.

◆ tryToWiden()

bool VPRecipeBuilder::tryToWiden	(	Instruction *	I,
		VPBasicBlock *	VPBB,
		VFRange &	Range
	)

Check if I can be widened within the given VF Range.

If I can be widened for Range.Start, check if the last recipe of VPBB can be extended to include I or else build a new VPWidenRecipe for it and append it to VPBB. Return true if I can be widened for Range.Start, false otherwise. Range.End may be decreased to ensure same decision from Range.Start to Range.End.

Definition at line 6509 of file LoopVectorize.cpp.

References llvm::MCID::Add, llvm::VPWidenRecipe::appendInstruction(), llvm::VPBasicBlock::appendRecipe(), assert(), llvm::Intrinsic::assume, llvm::VPBasicBlock::back(), llvm::MCID::Call, llvm::LoopVectorizationCostModel::CM_Scalarize, llvm::dyn_cast(), llvm::VPBasicBlock::empty(), llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), llvm::Instruction::getOpcode(), getVectorCallCost(), getVectorIntrinsicCost(), llvm::getVectorIntrinsicIDForCall(), llvm::Intrinsic::lifetime_end, llvm::Intrinsic::lifetime_start, llvm::SPII::Load, llvm::MCID::Select, llvm::Intrinsic::sideeffect, llvm::SPII::Store, llvm::InnerLoopVectorizer::TLI, llvm::InnerLoopVectorizer::TTI, and llvm::InnerLoopVectorizer::VF.

◆ tryToWidenMemory()

VPWidenMemoryInstructionRecipe * VPRecipeBuilder::tryToWidenMemory	(	Instruction *	I,
		VFRange &	Range,
		VPlanPtr &	Plan
	)

Check if is a memory instruction to be widened for Range.Start and potentially masked.

Such instructions are handled by a recipe that takes an additional VPInstruction for the mask.

Definition at line 6422 of file LoopVectorize.cpp.

References assert(), llvm::LoopVectorizationCostModel::CM_Interleave, llvm::LoopVectorizationCostModel::CM_Scalarize, llvm::LoopVectorizationCostModel::CM_Unknown, llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), llvm::Instruction::getParent(), llvm::LoopVectorizationLegality::isMaskRequired(), llvm::InnerLoopVectorizer::Legal, llvm::BitmaskEnumDetail::Mask(), and llvm::InnerLoopVectorizer::VF.

The documentation for this class was generated from the following files:

lib/Transforms/Vectorize/VPRecipeBuilder.h
lib/Transforms/Vectorize/LoopVectorize.cpp

Public Member Functions