Planner drives the vectorization process after having passed Legality checks. More...

#include "Transforms/Vectorize/LoopVectorizationPlanner.h"

Public Member Functions
	LoopVectorizationPlanner (Loop L, LoopInfo LI, const TargetLibraryInfo TLI, const TargetTransformInfo TTI, LoopVectorizationLegality *Legal, LoopVectorizationCostModel &CM)

VectorizationFactor	plan (bool OptForSize, unsigned UserVF)
	Plan how to best vectorize, return the best VF and its cost. More...

VectorizationFactor	planInVPlanNativePath (bool OptForSize, unsigned UserVF)
	Use the VPlan-native path to plan how to best vectorize, return the best VF and its cost. More...

void	setBestPlan (unsigned VF, unsigned UF)
	Finalize the best decision and dispose of all other VPlans. More...

void	executePlan (InnerLoopVectorizer &LB, DominatorTree *DT)
	Generate the IR code for the body of the vectorized loop according to the best selected VPlan. More...

void	printPlans (raw_ostream &O)

Static Public Member Functions
static bool	getDecisionAndClampRange (const std::function< bool(unsigned)> &Predicate, VFRange &Range)
	Test a `Predicate` on a `Range` of VF's. More...

Protected Member Functions
void	collectTriviallyDeadInstructions (SmallPtrSetImpl< Instruction *> &DeadInstructions)
	Collect the instructions from the original loop that would be trivially dead in the vectorized loop if generated. More...

void	buildVPlans (unsigned MinVF, unsigned MaxVF)
	Build VPlans for power-of-2 VF's between `MinVF` and `MaxVF` inclusive, according to the information gathered by Legal when it checked if it is legal to vectorize the loop. More...

Detailed Description

Planner drives the vectorization process after having passed Legality checks.

Definition at line 179 of file LoopVectorizationPlanner.h.

Constructor & Destructor Documentation

◆ LoopVectorizationPlanner()

llvm::LoopVectorizationPlanner::LoopVectorizationPlanner	(	Loop *	L,
		LoopInfo *	LI,
		const TargetLibraryInfo *	TLI,
		const TargetTransformInfo *	TTI,
		LoopVectorizationLegality *	Legal,
		LoopVectorizationCostModel &	CM
	)

inline

Definition at line 219 of file LoopVectorizationPlanner.h.

Member Function Documentation

◆ buildVPlans()

void LoopVectorizationPlanner::buildVPlans	(	unsigned	MinVF,
		unsigned	MaxVF
	)

protected

Build VPlans for power-of-2 VF's between MinVF and MaxVF inclusive, according to the information gathered by Legal when it checked if it is legal to vectorize the loop.

Build VPlans for the full range of feasible VF's = {MinVF, 2 * MinVF, 4 * MinVF, ..., MaxVF} by repeatedly building a VPlan for a sub-range of VF's starting at a given VF and extending it as much as possible.

Each vectorization decision can potentially shorten this sub-range during buildVPlan().

Definition at line 6310 of file LoopVectorize.cpp.

References llvm::VFRange::End, and llvm::InnerLoopVectorizer::VF.

◆ collectTriviallyDeadInstructions()

void LoopVectorizationPlanner::collectTriviallyDeadInstructions ( SmallPtrSetImpl< Instruction *> & DeadInstructions )

protected

Collect the instructions from the original loop that would be trivially dead in the vectorized loop if generated.

Definition at line 6199 of file LoopVectorize.cpp.

References addFastMathFlag(), llvm::all_of(), assert(), llvm::SmallVectorTemplateCommon< T >::begin(), llvm::InnerLoopVectorizer::Builder, C, llvm::IRBuilder< T, Inserter >::CreateAdd(), llvm::IRBuilder< T, Inserter >::CreateBinOp(), llvm::IRBuilder< T, Inserter >::CreateFMul(), llvm::IRBuilder< T, Inserter >::CreateMul(), llvm::dyn_cast(), llvm::SmallVectorTemplateCommon< T >::end(), llvm::SmallPtrSetImpl< PtrType >::end(), llvm::SmallPtrSetImpl< PtrType >::find(), llvm::ConstantInt::get(), llvm::ConstantFP::get(), llvm::InductionDescriptor::getCastInsts(), llvm::PHINode::getIncomingValueForBlock(), llvm::LoopVectorizationLegality::getInductionVars(), llvm::LoopBase< BlockT, LoopT >::getLoopLatch(), llvm::User::getOperand(), llvm::BasicBlock::getTerminator(), llvm::Value::getType(), llvm::InnerLoopVectorizer::Induction, llvm::SmallPtrSetImpl< PtrType >::insert(), llvm::Type::isFloatingPointTy(), llvm::Type::isVectorTy(), llvm::InnerLoopVectorizer::Legal, and llvm::InnerLoopVectorizer::OrigLoop.

◆ executePlan()

void LoopVectorizationPlanner::executePlan	(	InnerLoopVectorizer &	LB,
		DominatorTree *	DT
	)

Generate the IR code for the body of the vectorized loop according to the best selected VPlan.

Definition at line 6169 of file LoopVectorize.cpp.

References assert(), llvm::InnerLoopVectorizer::Builder, llvm::InnerLoopVectorizer::createVectorizedLoopSkeleton(), llvm::InnerLoopVectorizer::DT, llvm::InnerLoopVectorizer::fixVectorizedLoop(), llvm::InnerLoopVectorizer::getOrCreateTripCount(), llvm::InnerLoopVectorizer::LI, and llvm::InnerLoopVectorizer::VectorLoopValueMap.

Referenced by llvm::LoopVectorizePass::processLoop(), and processLoopInVPlanNativePath().

◆ getDecisionAndClampRange()

bool LoopVectorizationPlanner::getDecisionAndClampRange	(	const std::function< bool(unsigned)> &	Predicate,
		VFRange &	Range
	)

static

Test a Predicate on a Range of VF's.

Return the value of applying Predicate on Range.Start, possibly decreasing Range.End such that the returned value holds for the entire Range.

Definition at line 6291 of file LoopVectorize.cpp.

References assert(), llvm::VFRange::End, and llvm::VFRange::Start.

Referenced by llvm::VPRecipeBuilder::handleReplication(), llvm::VPRecipeBuilder::tryToInterleaveMemory(), llvm::VPRecipeBuilder::tryToOptimizeInduction(), llvm::VPRecipeBuilder::tryToWiden(), and llvm::VPRecipeBuilder::tryToWidenMemory().

◆ plan()

VectorizationFactor LoopVectorizationPlanner::plan	(	bool	OptForSize,
		unsigned	UserVF
	)

Plan how to best vectorize, return the best VF and its cost.

Definition at line 6106 of file LoopVectorize.cpp.

References assert(), llvm::dbgs(), llvm::LoopBase< BlockT, LoopT >::empty(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::Optional< T >::getValue(), llvm::Optional< T >::hasValue(), llvm::isPowerOf2_32(), LLVM_DEBUG, llvm::InnerLoopVectorizer::OrigLoop, llvm::InnerLoopVectorizer::TTI, and useMaskedInterleavedAccesses().

Referenced by llvm::LoopVectorizePass::processLoop().

◆ planInVPlanNativePath()

VectorizationFactor LoopVectorizationPlanner::planInVPlanNativePath	(	bool	OptForSize,
		unsigned	UserVF
	)

Use the VPlan-native path to plan how to best vectorize, return the best VF and its cost.

Definition at line 6070 of file LoopVectorize.cpp.

References assert(), llvm::dbgs(), llvm::LoopBase< BlockT, LoopT >::empty(), EnableVPlanNativePath, llvm::isPowerOf2_32(), LLVM_DEBUG, llvm::InnerLoopVectorizer::OrigLoop, and VPlanBuildStressTest.

Referenced by processLoopInVPlanNativePath().

◆ printPlans()

void llvm::LoopVectorizationPlanner::printPlans ( raw_ostream & O )

inline

Definition at line 239 of file LoopVectorizationPlanner.h.

References function.

◆ setBestPlan()

void LoopVectorizationPlanner::setBestPlan	(	unsigned	VF,
		unsigned	UF
	)

Finalize the best decision and dispose of all other VPlans.

Definition at line 6157 of file LoopVectorize.cpp.

References assert(), llvm::dbgs(), llvm::erase_if(), LLVM_DEBUG, llvm::InnerLoopVectorizer::UF, and llvm::InnerLoopVectorizer::VF.